-- Logs begin at Mon 2022-10-03 12:49:54 UTC, end at Mon 2022-10-03 13:59:00 UTC. -- Oct 03 13:14:49.302436 dv-cinder-860090-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 03 13:14:49.315775 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 3 13:14:49 2022] *** Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: nodename: dv-cinder-860090-1-iscsi Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: machine: x86_64 Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: clock source: unix Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: pcre jit disabled Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: detected number of CPU cores: 4 Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: current working directory: / Oct 03 13:14:49.319440 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: detected binary path: /usr/bin/uwsgi-core Oct 03 13:14:49.320488 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: your processes number limit is 47680 Oct 03 13:14:49.320488 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: your memory page size is 4096 bytes Oct 03 13:14:49.320488 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: detected max file descriptor number: 2048 Oct 03 13:14:49.320488 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: lock engine: pthread robust mutexes Oct 03 13:14:49.320488 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: thunder lock: enabled Oct 03 13:14:49.320488 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 03 13:14:49.320488 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 03 13:14:49.365542 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: Python main interpreter initialized at 0x55eddbb429c0 Oct 03 13:14:49.365542 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: python threads support enabled Oct 03 13:14:49.365542 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: your server socket listen backlog is limited to 100 connections Oct 03 13:14:49.365542 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: your mercy for graceful operations on workers is 90 seconds Oct 03 13:14:49.366145 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: mapped 403077 bytes (393 KB) for 2 cores Oct 03 13:14:49.366145 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: *** Operational MODE: preforking *** Oct 03 13:14:49.366364 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: *** uWSGI is running in multiple interpreter mode *** Oct 03 13:14:49.366364 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: spawned uWSGI master process (pid: 81412) Oct 03 13:14:49.366560 dv-cinder-860090-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 03 13:14:49.368818 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: spawned uWSGI worker 1 (pid: 81423, cores: 1) Oct 03 13:14:49.368818 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: spawned uWSGI worker 2 (pid: 81424, cores: 1) Oct 03 13:14:49.368818 dv-cinder-860090-1-iscsi devstack@keystone.service[81412]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 03 13:14:54.030644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 13:14:54.030644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: from cryptography.utils import int_from_bytes Oct 03 13:14:54.031950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 13:14:54.031950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: from cryptography.utils import int_from_bytes Oct 03 13:14:54.092619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 03 13:14:54.092920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 03 13:14:54.093069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] command line args: [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 03 13:14:54.093257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 03 13:14:54.093443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ================================================================================ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 03 13:14:54.093741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] admin_token = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.094003 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_dir = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.094201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.094392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_source = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.094580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] debug = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.094982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.095260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.095386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.095666 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.095865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.096109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.096355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_config_append = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.096636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.096761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_dir = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.096885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.098453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.098453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.098453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.098453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.098453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.098453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.098453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.099987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.099987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.099987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.099987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.100578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.100578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.100894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.101150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.101721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.101932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] publish_errors = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.102144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.102393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.102590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.102833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.103079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.103326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] standard_threads = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.103568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.103823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.104016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.104185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_journal = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.104359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_json = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.104517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_stderr = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.104676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_syslog = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.104908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:54.105181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.105427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.105736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.105986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.106245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.106493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.106749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.106995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.107250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.107531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.107753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.107996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.108251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.108497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.108753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.109011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.109255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.109547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.109813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.110069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.110318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.110582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.110856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.111116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.111376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.111615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.111853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.112091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.112335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.112591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.112846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.113083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.113875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.113875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.113875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.114414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.114414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.114759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.114759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.116453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.116725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.116979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.117231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.117503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.117759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.118029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.118274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.118526 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.118793 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.119007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.119259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.119522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.119764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.120012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.120258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.120651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.120839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.121025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.121210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.121536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.121749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.121961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.122204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.122476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.122722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.122961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.123206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.123732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.123732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.124051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.124219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.124442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.124690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.124932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.125175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.125435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.125710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.125956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.126210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.126454 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.128621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.130290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.130290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.130290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.131615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.132195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.132719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.132980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.133237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.133521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.133785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.134034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.134294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.134548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.134820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.135072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.135345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.135624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.135879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.136113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.external = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.136351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.136662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.136908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.137151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.password = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.137391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.token = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.137677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.137925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.138186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.138428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.138687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.138945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.139187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.139435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.139693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.139953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.140211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.140465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.140718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.140965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.141223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.141509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.141767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.142031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.142330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.142589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.142865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.143118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.143386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.143632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.143946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.144132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.144363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.144603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.144861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.145110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.145375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.145653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.145919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.146521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.147301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.147566 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.147834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.148095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.148347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.148762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.148942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.149146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.149343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.149573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.149842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.150084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.150323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.151806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.151806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.151806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.151806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.151806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.151806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.153070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.153070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.153070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.153070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.153070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.154027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.154027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.154027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.154027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.154676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.154676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.155002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.155002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.155351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.155531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.155725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.155955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.156204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.156443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.156692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.156938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.157194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.157882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.157882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.158308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.158487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.158678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.158856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.159048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.159309 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.159560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.159807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.160054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.160293 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.160549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.160816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.161057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.161315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.161610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.162793 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.163057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.163303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.163554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.163804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.164055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.164297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.164561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.164821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.165078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.165333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.165621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.165864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.166124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.166370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.166615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.166861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.167104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.167361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.167620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.167866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.168131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.168369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.170662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.172230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.172230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.172230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.172840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.173159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.173524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.173721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.173963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.174276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.174466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.174732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.174977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.175240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.175504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.175764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.176022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.176280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.176538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.176797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.177057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.177299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.178545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.179113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.179387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.179651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.180110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.180365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.180634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.180899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.181164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.181431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.181713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.181970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.182234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.182485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.182728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.182968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.183209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.183473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.183748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.184005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.184255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.184498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.184740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.184985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.185232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.185545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.185776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.186045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.186302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.186553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.186805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.187060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.187317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.187560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.187809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.188051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.188306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.188625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.188878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.189342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.189342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.189752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.189940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.190121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.190378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.190594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.190842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.191087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.191331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.191579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.191833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.192079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.192329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.192577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.192840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.193225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.194297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.194544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.194806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.195057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:54.195241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 03 13:14:54.232874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.233130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.233335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.244796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.279271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.279540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.345105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.404436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.404680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.588145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.588380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.588599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.588825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.589058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.589477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.620220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.620407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.620604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.620836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.621065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.621288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.621538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.639758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.655903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.656208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.656446 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.656679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.656915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.657143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.657370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:54.677133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.677564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 13:14:54.678140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.678352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 13:14:54.678640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.678913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.679187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.679459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.690445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.690745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.691027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.691324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.701974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.707293 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.707643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.708158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.709333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.709870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.710144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.710430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.710771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.711267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.711761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.712249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.740533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.742463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.742978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.748827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.749158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.749469 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.757803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.758113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.764639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.765148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.765748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.766172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.766463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.766748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.767029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.767441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.790853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.791131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.795276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.800040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.800359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.805903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.808998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.812445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.812805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.813296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.813852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.814233 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.814531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.832484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.832984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.833509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.834071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.844878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.847546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.850043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.850524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.854374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.854706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.855196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.855694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.856335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.870544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.870782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.871150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.871667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.872365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.872736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.873036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.891567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.895317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.898861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.900352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.901872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.902168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.902589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.910776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.913010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.918253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.918891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.919370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.919837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.930264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.930601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.931743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.938758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:54.939058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.939385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.939875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.940382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.940982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.941323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.941643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.941927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.942226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.942518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.943048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.944198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.944629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:54.985907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55eddbb429c0 pid: 81424 (default app) Oct 03 13:14:54.991815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cba0ec-dd54-46ec-921e-a0ac930ee123 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:14:54.992314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cba0ec-dd54-46ec-921e-a0ac930ee123 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:14:54.992814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cba0ec-dd54-46ec-921e-a0ac930ee123 None None] PATH_INFO: `/v3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:14:54.998616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Mon Oct 3 13:14:54 2022] GET /identity/v3/ => generated 255 bytes in 12 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:14:55.036277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 13:14:55.036277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: from cryptography.utils import int_from_bytes Oct 03 13:14:55.037249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 03 13:14:55.037249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: from cryptography.utils import int_from_bytes Oct 03 13:14:55.088094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 03 13:14:55.088302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 03 13:14:55.088506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] command line args: [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 03 13:14:55.088779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 03 13:14:55.088959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ================================================================================ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 03 13:14:55.089216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] admin_token = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.089479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] config_dir = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.089670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.089854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] config_source = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.090035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] debug = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.090635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.090635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.090635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.090870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.091430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.091430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.091552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_config_append = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.091752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.091842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_dir = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.092017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_file = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.092254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.092502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.092747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.092991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.093169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.093345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.093623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.095748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.095748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.095748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.095748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.095748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.095748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.095748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.095748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.096301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.096301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] publish_errors = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.096444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.096596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.096774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.097009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.097240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.097507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] standard_threads = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.098143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.098143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.098143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.098487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_journal = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.098487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_json = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.099094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_stderr = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.099094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] use_syslog = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.099378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 03 13:14:55.099466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.099696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.099967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.100208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.100493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.100687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.100937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.101244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.101502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.101692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.101947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.102185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.102435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.102673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.102920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.103166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.103403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.103658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.103914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.104162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.104400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.104648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.105072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.105211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.105547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.105700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.105869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.106335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.106784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.107050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.107289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.107523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.107915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.108087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.108340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.108776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.108776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.109168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.109168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.109538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.109734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.111476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.111476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.111476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.111476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.113205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.113205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.113205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.113205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.113205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.113205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.113205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.113205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.117109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.118326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.119456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.119456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.119456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.119456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.119456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.119456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.120189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.120189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.120189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.120595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.120595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.120868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.121000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.121210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.121469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.121753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.121996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.122246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.122520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.122764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.123078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.123339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.123577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.123815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.124048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.124278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.124509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.124749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.124981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.125224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.125554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.135108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.135108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.135108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.135108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.135772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.135772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.external = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.135772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.137110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.137110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.137110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.password = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.137110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] auth.token = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.137110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.137860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.137860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.137860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.138242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.138242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.138490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.138659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.138910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.139166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.139424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.139690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.139936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.140178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.140436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.140755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.141000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.141258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.141687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.141946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.142216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.142465 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.142740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.142983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.143223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.143474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.143724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.143959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.144211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.144455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.144712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.144952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.145209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.145481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.150296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.151469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.152568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.152568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.152568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.152568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.152568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.152568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.152568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.153418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.153418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.153418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.153418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.154054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.154054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.154302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.154466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.154719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.154967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.155207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.155446 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.155680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.155986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.156245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.156495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.156737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.156977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.157225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.157653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.157893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.158132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.158508 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.158763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.159003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.159247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.159486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.159721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.159966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.160229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.160463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.160712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.160961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.161213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.163202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.163202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.163202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.163646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.164116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.164564 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.165055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.165560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.166057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.166519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.166999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.167412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.167905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.168360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.168810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.169266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.169835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.170307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.170784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.171237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.171726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.172176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.172645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.173099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.173579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.174034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.174510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.174970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.175420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.175834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.176296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.176772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.177201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.177769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.178433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.178915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.179362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.179830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.180282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.180736 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.181186 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] role.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.181682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] role.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.182132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.182603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.183071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.183546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.184016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.184501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.184968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.185443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.185899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.186342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.186803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.187247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.187720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.188184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.188955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.189398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.189901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.190377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.190851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.191320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.191766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.192252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.192720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.193810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.194253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.194690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.195131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.195596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.196082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.196510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.196965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.197420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.197842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.198288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.198741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.199202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.199652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.200132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.200603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.201063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.201557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.202020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.202486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.202939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.203389 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.203833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.204294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.204812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.205246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.205741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.206242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.206722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.207164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.207635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.208069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.208507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.209595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.210056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.210503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.210955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.211428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.211883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.212347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.212806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.213277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.213753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.214221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.214659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.215181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.215568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 03 13:14:55.215881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81423) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 03 13:14:55.266100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.266484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.266874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.282872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.319418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.319819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.387526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.456760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.457166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.539405 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.539818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.540227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.540662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.541085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.541533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.567884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.568396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.568786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.569180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.569600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.569993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.570391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.713377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.718639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.719011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.719387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.719753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.720119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.720481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.720858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81423) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 03 13:14:55.740734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.741085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 13:14:55.741686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.742028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 03 13:14:55.742473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.742912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.743364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.743770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.771032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.771714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.772213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.772711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.787238 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.793000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.793527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.794020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.794530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.795035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.795525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.796023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.796539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.797045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.797581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.798117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.845380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.848268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.848734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.859111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.859644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.860147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.874096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.874629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.883431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.883944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.884437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.884957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.885442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.886164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.886635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.887152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.919580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.920076 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.928452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.934649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.935172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.943775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.949975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.956077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.956600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.957057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.957739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.958258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.958810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.995087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:55.995538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.996016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:55.996493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.010399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.013547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.016511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.017014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.022606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:56.023117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.023599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.024116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.024645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.048155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:56.048700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.049160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.049696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.050215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.050732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.051260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.085277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:56.093856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:56.099362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.104183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.107050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:56.107562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.108067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.124518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.127337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:56.133879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.140503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.141034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.141565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.152935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:56.153235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.153549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.160368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81423) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 03 13:14:56.160653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.160940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.161225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.161520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.161807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.162084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.162368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.162655 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.162945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.163221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.163494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.163762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.164021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81423) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 03 13:14:56.197642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55eddbb429c0 pid: 81423 (default app) Oct 03 13:15:02.086215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30469125-c44e-4b76-a08f-71389d5c38ae None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:02.086901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30469125-c44e-4b76-a08f-71389d5c38ae None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:02.086901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30469125-c44e-4b76-a08f-71389d5c38ae None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:02.088092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 2/2] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:02.096336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:02.097027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:02.097027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:02.173291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: INFO dbcounter [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] Registered counter for database keystone Oct 03 13:15:02.194731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG oslo_db.sqlalchemy.engines [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81423) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 03 13:15:02.201678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writer thread running {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 03 13:15:02.396639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG passlib.handlers.bcrypt [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81423) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 03 13:15:02.397825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG passlib.handlers.bcrypt [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81423) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 03 13:15:02.458172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:02.461012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:02.627999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1/3] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 535 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 03 13:15:02.942233 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30469125-c44e-4b76-a08f-71389d5c38ae None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:03.017649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: INFO dbcounter [None req-30469125-c44e-4b76-a08f-71389d5c38ae None None] Registered counter for database keystone Oct 03 13:15:03.040681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG oslo_db.sqlalchemy.engines [None req-30469125-c44e-4b76-a08f-71389d5c38ae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 03 13:15:03.047135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writer thread running {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 03 13:15:03.173907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30469125-c44e-4b76-a08f-71389d5c38ae None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:03.179624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c822840-1f37-45e7-915f-700ca9e914c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:03.180280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c822840-1f37-45e7-915f-700ca9e914c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:03.180570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c822840-1f37-45e7-915f-700ca9e914c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:03.180853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c822840-1f37-45e7-915f-700ca9e914c4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:03.190218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c822840-1f37-45e7-915f-700ca9e914c4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:03.246651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c822840-1f37-45e7-915f-700ca9e914c4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 03 13:15:03.246924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c822840-1f37-45e7-915f-700ca9e914c4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:03.259185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 3/4] 10.222.0.58 () {58 vars in 1324 bytes} [Mon Oct 3 13:15:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 13:15:03.265272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:03.281329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358558-1ee9-4c58-b680-4f37b164420f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:03.285937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29475f6e-373e-4205-ba42-933ab31177a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:03.287744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29475f6e-373e-4205-ba42-933ab31177a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:03.288021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29475f6e-373e-4205-ba42-933ab31177a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:03.288294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29475f6e-373e-4205-ba42-933ab31177a9 admin admin] PATH_INFO: `/v3/projects/cf1b6fdf131f4a598a5705727a85c56c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:03.289860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29475f6e-373e-4205-ba42-933ab31177a9 admin admin] RBAC: Authorizing `identity:get_project(project_id=cf1b6fdf131f4a598a5705727a85c56c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:03.343806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING keystone.common.rbac_enforcer.enforcer [None req-29475f6e-373e-4205-ba42-933ab31177a9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 03 13:15:03.344168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29475f6e-373e-4205-ba42-933ab31177a9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:03.349822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 2/5] 10.222.0.58 () {56 vars in 1207 bytes} [Mon Oct 3 13:15:03 2022] GET /identity/v3/projects/cf1b6fdf131f4a598a5705727a85c56c => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:05.663335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacbb8c-f610-476c-9f6a-838cffbf0d5c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:05.663862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacbb8c-f610-476c-9f6a-838cffbf0d5c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:05.663862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacbb8c-f610-476c-9f6a-838cffbf0d5c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:05.667962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 4/6] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:05.674134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df05ee-16e4-4d67-8adb-a159ab509e0b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:05.674596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df05ee-16e4-4d67-8adb-a159ab509e0b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:05.674699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df05ee-16e4-4d67-8adb-a159ab509e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:05.696544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a2df05ee-16e4-4d67-8adb-a159ab509e0b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:05.698157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a2df05ee-16e4-4d67-8adb-a159ab509e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:05.706205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 3/7] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 03 13:15:05.976768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacbb8c-f610-476c-9f6a-838cffbf0d5c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:05.988965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacbb8c-f610-476c-9f6a-838cffbf0d5c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:05.994070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8367a-f433-4370-89c8-18f6fc1370b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:05.994685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8367a-f433-4370-89c8-18f6fc1370b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:05.994968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8367a-f433-4370-89c8-18f6fc1370b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:05.995241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8367a-f433-4370-89c8-18f6fc1370b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:06.003444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8367a-f433-4370-89c8-18f6fc1370b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:06.005816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db8367a-f433-4370-89c8-18f6fc1370b2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:06.015176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 5/8] 10.222.0.58 () {58 vars in 1324 bytes} [Mon Oct 3 13:15:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:15:06.021128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df05ee-16e4-4d67-8adb-a159ab509e0b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:06.033082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df05ee-16e4-4d67-8adb-a159ab509e0b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:06.037839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20966d-3b05-4c25-ac4f-2d13e53c4515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:06.038493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20966d-3b05-4c25-ac4f-2d13e53c4515 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:06.038783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20966d-3b05-4c25-ac4f-2d13e53c4515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:06.039065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20966d-3b05-4c25-ac4f-2d13e53c4515 admin admin] PATH_INFO: `/v3/users/72593536090c47589cfad801ff3da440` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:06.040416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20966d-3b05-4c25-ac4f-2d13e53c4515 admin admin] RBAC: Authorizing `identity:get_user(user_id=72593536090c47589cfad801ff3da440)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:06.042605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20966d-3b05-4c25-ac4f-2d13e53c4515 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:06.044199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 4/9] 10.222.0.58 () {56 vars in 1198 bytes} [Mon Oct 3 13:15:06 2022] GET /identity/v3/users/72593536090c47589cfad801ff3da440 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:08.416016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aebb17-071e-405e-897e-6cf58db63346 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:08.416598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aebb17-071e-405e-897e-6cf58db63346 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:08.416598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aebb17-071e-405e-897e-6cf58db63346 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:08.418173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 6/10] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:08.423581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5360f2-b66a-46e7-92c5-e47fd1858ca7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:08.424092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5360f2-b66a-46e7-92c5-e47fd1858ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:08.424226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5360f2-b66a-46e7-92c5-e47fd1858ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:08.442598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ea5360f2-b66a-46e7-92c5-e47fd1858ca7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:08.443803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ea5360f2-b66a-46e7-92c5-e47fd1858ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:08.530696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 5/11] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:08.787460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74604-f537-4f2f-a487-db2a671c92b2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:08.787838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74604-f537-4f2f-a487-db2a671c92b2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:08.788212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad74604-f537-4f2f-a487-db2a671c92b2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:08.789669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 7/12] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:08.797723 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d549f-4e05-4368-8979-abe121039795 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:08.798134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d549f-4e05-4368-8979-abe121039795 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:08.798515 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d549f-4e05-4368-8979-abe121039795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:08.807803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74604-f537-4f2f-a487-db2a671c92b2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:08.825814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-936d549f-4e05-4368-8979-abe121039795 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:08.828045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-936d549f-4e05-4368-8979-abe121039795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:08.859732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:08.941023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 6/13] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:08.976645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad74604-f537-4f2f-a487-db2a671c92b2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:09.014025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea75a-213d-42e7-b075-2bc37f5ae923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:09.014902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea75a-213d-42e7-b075-2bc37f5ae923 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:09.015186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea75a-213d-42e7-b075-2bc37f5ae923 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:09.015462 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aea75a-213d-42e7-b075-2bc37f5ae923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:09.027871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aea75a-213d-42e7-b075-2bc37f5ae923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-52aea75a-213d-42e7-b075-2bc37f5ae923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:09.037703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:09.039221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:09.040224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 8/14] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:15:09.044880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d549f-4e05-4368-8979-abe121039795 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:09.149050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d549f-4e05-4368-8979-abe121039795 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:09.185033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:09.185692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:09.185980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:09.186261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:09.186941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:09.187253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:09.188665 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:09.201332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 7/15] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:09.207080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea75a-213d-42e7-b075-2bc37f5ae923 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:09.243143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:09.316546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aea75a-213d-42e7-b075-2bc37f5ae923 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:09.354979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76650554-ffa0-403c-8b62-6342a79534d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:09.355633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76650554-ffa0-403c-8b62-6342a79534d7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:09.355917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76650554-ffa0-403c-8b62-6342a79534d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:09.356194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76650554-ffa0-403c-8b62-6342a79534d7 None admin] PATH_INFO: `/v3/users/72593536090c47589cfad801ff3da440` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:09.357822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76650554-ffa0-403c-8b62-6342a79534d7 None admin] RBAC: Authorizing `identity:get_user(user_id=72593536090c47589cfad801ff3da440)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:09.359300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76650554-ffa0-403c-8b62-6342a79534d7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:09.360895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 9/16] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:09 2022] GET /identity/v3/users/72593536090c47589cfad801ff3da440 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:09.366969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:09.384150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:09.469229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76650554-ffa0-403c-8b62-6342a79534d7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:09.473206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8b609-ca74-48d4-9c5b-b58e68c04764 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:09.523565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6233ec3-b41f-498e-822a-6755cf93c4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:09.524499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233ec3-b41f-498e-822a-6755cf93c4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:09.524989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233ec3-b41f-498e-822a-6755cf93c4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:09.525489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6233ec3-b41f-498e-822a-6755cf93c4b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:09.527173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6233ec3-b41f-498e-822a-6755cf93c4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:09.528895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6233ec3-b41f-498e-822a-6755cf93c4b5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:09.531070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 8/17] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:09.537401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6233ec3-b41f-498e-822a-6755cf93c4b5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:09.619315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76650554-ffa0-403c-8b62-6342a79534d7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:09.621172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:09.639700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:09.664929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e0c26-091a-45f6-a0ee-cd701e28f086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:09.665593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e0c26-091a-45f6-a0ee-cd701e28f086 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:09.665885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e0c26-091a-45f6-a0ee-cd701e28f086 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:09.666169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e0c26-091a-45f6-a0ee-cd701e28f086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:09.672252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6233ec3-b41f-498e-822a-6755cf93c4b5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:09.730174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:09.730820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:09.731086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:09.731365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:09.732102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.domain.id=default, user.id=72593536090c47589cfad801ff3da440) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:09.732402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:09.733940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:09.742215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e0c26-091a-45f6-a0ee-cd701e28f086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:09.742452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 9/18] 10.222.0.58 () {56 vars in 1322 bytes} [Mon Oct 3 13:15:09 2022] GET /identity/v3/role_assignments?user.id=72593536090c47589cfad801ff3da440&scope.domain.id=default&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 233 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:09.744091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e0c26-091a-45f6-a0ee-cd701e28f086 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:09.872912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 10/19] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 13:15:09.879174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:09.997271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754defec-8fdb-49c8-b527-43fdc2a5f40a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:10.033885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ccce7-f8fe-454c-bf41-b4a914472b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:10.034535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ccce7-f8fe-454c-bf41-b4a914472b06 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:10.034823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ccce7-f8fe-454c-bf41-b4a914472b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:10.035099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ccce7-f8fe-454c-bf41-b4a914472b06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:10.046805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ccce7-f8fe-454c-bf41-b4a914472b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:10.048381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ccce7-f8fe-454c-bf41-b4a914472b06 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:10.057072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-275ccce7-f8fe-454c-bf41-b4a914472b06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:15:10.070755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:10.071792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:15:10.072595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:15:10.072595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:15:10.072595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:15:10.072595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 10/20] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:15:10.078357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e0c26-091a-45f6-a0ee-cd701e28f086 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:10.080723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:10.184964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e0c26-091a-45f6-a0ee-cd701e28f086 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:10.222992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b88c53-b135-4526-9cb3-39037cafb005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:10.223665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88c53-b135-4526-9cb3-39037cafb005 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:10.223951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88c53-b135-4526-9cb3-39037cafb005 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:10.224229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b88c53-b135-4526-9cb3-39037cafb005 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:10.224911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b88c53-b135-4526-9cb3-39037cafb005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:10.225221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b88c53-b135-4526-9cb3-39037cafb005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:10.226738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b88c53-b135-4526-9cb3-39037cafb005 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:10.251010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 11/21] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:13.024027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0746d-1a56-40a6-8127-45e83546527d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:13.025092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0746d-1a56-40a6-8127-45e83546527d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:13.025636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0746d-1a56-40a6-8127-45e83546527d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:13.027275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 11/22] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:13.033203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625130d-d961-4e0b-b76a-73b97884bc37 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:13.034303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625130d-d961-4e0b-b76a-73b97884bc37 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:13.034815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625130d-d961-4e0b-b76a-73b97884bc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:13.059195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG passlib.handlers.bcrypt [None req-3625130d-d961-4e0b-b76a-73b97884bc37 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81424) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 03 13:15:13.059922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG passlib.handlers.bcrypt [None req-3625130d-d961-4e0b-b76a-73b97884bc37 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81424) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 03 13:15:13.125696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3625130d-d961-4e0b-b76a-73b97884bc37 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:13.127678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3625130d-d961-4e0b-b76a-73b97884bc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:13.230555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:13.245246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 12/23] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:13.686087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8957aa8-9071-4281-84d6-6d9e1195427d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:13.686804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8957aa8-9071-4281-84d6-6d9e1195427d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:13.687509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8957aa8-9071-4281-84d6-6d9e1195427d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:13.689178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 12/24] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:13.700032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389b307-ffe1-4abd-8bb1-0a3d4867378b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:13.700687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389b307-ffe1-4abd-8bb1-0a3d4867378b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:13.701167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389b307-ffe1-4abd-8bb1-0a3d4867378b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:13.728843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9389b307-ffe1-4abd-8bb1-0a3d4867378b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:13.730546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9389b307-ffe1-4abd-8bb1-0a3d4867378b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:13.833911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 13/25] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:13.871594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5fe73-08b0-40bf-917d-a4a4e5905c58 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:13.872282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5fe73-08b0-40bf-917d-a4a4e5905c58 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:13.872778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5fe73-08b0-40bf-917d-a4a4e5905c58 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:13.874548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 13/26] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:13.881578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3e75-b4cf-448d-b829-cb944e05b645 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:13.882447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3e75-b4cf-448d-b829-cb944e05b645 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:13.882919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b3e75-b4cf-448d-b829-cb944e05b645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:13.908557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-563b3e75-b4cf-448d-b829-cb944e05b645 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:13.910261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-563b3e75-b4cf-448d-b829-cb944e05b645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:14.051443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 14/27] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:14.061702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bfc9f-cd9c-4642-a3a5-d6daf4defbad None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:14.062610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bfc9f-cd9c-4642-a3a5-d6daf4defbad None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:14.063082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bfc9f-cd9c-4642-a3a5-d6daf4defbad None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:14.064663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 14/28] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:14.071187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1917-d393-4faf-9283-16b719b06bc5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:14.071489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1917-d393-4faf-9283-16b719b06bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:14.071778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1917-d393-4faf-9283-16b719b06bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:14.109672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d7bc1917-d393-4faf-9283-16b719b06bc5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:14.111872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d7bc1917-d393-4faf-9283-16b719b06bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:14.219353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:14.228327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bfc9f-cd9c-4642-a3a5-d6daf4defbad None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:14.267984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 15/29] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:14.391996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bfc9f-cd9c-4642-a3a5-d6daf4defbad None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:14.446398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381e899-f46a-4a3a-bcea-961b5be27e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:14.447124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381e899-f46a-4a3a-bcea-961b5be27e79 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:14.447410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381e899-f46a-4a3a-bcea-961b5be27e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:14.447686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381e899-f46a-4a3a-bcea-961b5be27e79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:14.467571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5381e899-f46a-4a3a-bcea-961b5be27e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5381e899-f46a-4a3a-bcea-961b5be27e79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:14.475150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:14.476638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:14.478023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:15:14.481927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 15/30] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:15:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:14.484039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc1917-d393-4faf-9283-16b719b06bc5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:14.576163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381e899-f46a-4a3a-bcea-961b5be27e79 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:14.687984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:14.744153 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381e899-f46a-4a3a-bcea-961b5be27e79 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:14.786809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a3931-f3eb-4700-abde-53b7adb129e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:14.787272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a3931-f3eb-4700-abde-53b7adb129e9 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:14.787602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a3931-f3eb-4700-abde-53b7adb129e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:14.789490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a3931-f3eb-4700-abde-53b7adb129e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:14.790223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a3931-f3eb-4700-abde-53b7adb129e9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:14.791661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a3931-f3eb-4700-abde-53b7adb129e9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:14.806573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc1917-d393-4faf-9283-16b719b06bc5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:14.886825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:14.888352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:14.889121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:14.889940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:14.891490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:14.892492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:14.896242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:14.932594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 16/31] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:14.933507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:14.996559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:15.066595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac8f23-015a-4b6e-b97c-a9f2a12eefae None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:15.111887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 16/32] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Oct 3 13:15:14 2022] POST /identity/v3/roles => generated 225 bytes in 537 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:15:15.115002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a3931-f3eb-4700-abde-53b7adb129e9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:15.128095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:15.129473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:15.130177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:15.130791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:15.193208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:15.203185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:15:15.243686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:15.245352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:15:15.246875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:15:15.246875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:15:15.246875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:15.246875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 17/33] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:15.247246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:15.354837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a3931-f3eb-4700-abde-53b7adb129e9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:15.414005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:15.449633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed1516-9169-4f59-b1db-a500c9f026c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:15.451253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1516-9169-4f59-b1db-a500c9f026c8 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:15.451745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1516-9169-4f59-b1db-a500c9f026c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:15.452304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed1516-9169-4f59-b1db-a500c9f026c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:15.453756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed1516-9169-4f59-b1db-a500c9f026c8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:15.456757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed1516-9169-4f59-b1db-a500c9f026c8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:15.492618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 17/34] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Oct 3 13:15:15 2022] POST /identity/v3/roles => generated 219 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:15:15.495224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed1516-9169-4f59-b1db-a500c9f026c8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:15.505855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:15.518049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e2861-6f26-4302-95b3-a89ae994ef61 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:15.583947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121fc66-45a5-43b6-9946-47b1e858cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:15.584702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121fc66-45a5-43b6-9946-47b1e858cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:15.584998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121fc66-45a5-43b6-9946-47b1e858cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:15.585283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121fc66-45a5-43b6-9946-47b1e858cbb8 None admin] PATH_INFO: `/v3/users/72593536090c47589cfad801ff3da440` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:15.587741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121fc66-45a5-43b6-9946-47b1e858cbb8 None admin] RBAC: Authorizing `identity:get_user(user_id=72593536090c47589cfad801ff3da440)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:15.589312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121fc66-45a5-43b6-9946-47b1e858cbb8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:15.592129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 18/35] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:15 2022] GET /identity/v3/users/72593536090c47589cfad801ff3da440 => generated 247 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:15.593586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f243-d6f8-4062-8c59-a7538f7b888a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:15.593971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f243-d6f8-4062-8c59-a7538f7b888a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:15.594255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6f243-d6f8-4062-8c59-a7538f7b888a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:15.598171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 19/36] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:15.602997 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f243-d6f8-4062-8c59-a7538f7b888a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:15.688932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed1516-9169-4f59-b1db-a500c9f026c8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:15.736284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3e7b7-d436-4172-a7ad-82161f636df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:15.737375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3e7b7-d436-4172-a7ad-82161f636df3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:15.737939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3e7b7-d436-4172-a7ad-82161f636df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:15.738440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3e7b7-d436-4172-a7ad-82161f636df3 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:15.740270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3e7b7-d436-4172-a7ad-82161f636df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:15.740809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3e7b7-d436-4172-a7ad-82161f636df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:15.741291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6f243-d6f8-4062-8c59-a7538f7b888a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:15.743584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3e7b7-d436-4172-a7ad-82161f636df3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:15.784324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 18/37] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:15.784324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d86453-af5a-4b71-a835-3b00d0925780 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:15.784324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d86453-af5a-4b71-a835-3b00d0925780 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:15.784324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d86453-af5a-4b71-a835-3b00d0925780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:15.788344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086bc54-f386-4277-b1f6-5c929d08cf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:15.789039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086bc54-f386-4277-b1f6-5c929d08cf57 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:15.789331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086bc54-f386-4277-b1f6-5c929d08cf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:15.789663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086bc54-f386-4277-b1f6-5c929d08cf57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:15.790976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086bc54-f386-4277-b1f6-5c929d08cf57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:15.792466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086bc54-f386-4277-b1f6-5c929d08cf57 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:15.794334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 20/38] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:15.795464 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfee9-f0be-4f1f-8d4f-17b789244f92 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:15.795851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfee9-f0be-4f1f-8d4f-17b789244f92 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:15.796139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbfee9-f0be-4f1f-8d4f-17b789244f92 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:15.797595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 21/39] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:15.798198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbfee9-f0be-4f1f-8d4f-17b789244f92 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:15.821410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a1d86453-af5a-4b71-a835-3b00d0925780 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:15.823020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a1d86453-af5a-4b71-a835-3b00d0925780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:15.849334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:15.877782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:15.942084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 19/40] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:15.943511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d86453-af5a-4b71-a835-3b00d0925780 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:15.975878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbfee9-f0be-4f1f-8d4f-17b789244f92 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:16.038743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603b41c-af63-4353-ac3e-98071db14c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:16.040105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b41c-af63-4353-ac3e-98071db14c9f None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:16.041166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b41c-af63-4353-ac3e-98071db14c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:16.041888 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b41c-af63-4353-ac3e-98071db14c9f None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:16.043264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603b41c-af63-4353-ac3e-98071db14c9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:16.046210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603b41c-af63-4353-ac3e-98071db14c9f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:16.086962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d86453-af5a-4b71-a835-3b00d0925780 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:16.155701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfacc0-75ca-48c3-938a-3ff71b7a7c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:16.156842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfacc0-75ca-48c3-938a-3ff71b7a7c49 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:16.157349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfacc0-75ca-48c3-938a-3ff71b7a7c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:16.157862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfacc0-75ca-48c3-938a-3ff71b7a7c49 None admin] PATH_INFO: `/v3/domains/default/users/72593536090c47589cfad801ff3da440/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:16.161978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfacc0-75ca-48c3-938a-3ff71b7a7c49 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=72593536090c47589cfad801ff3da440, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:16.164682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccfacc0-75ca-48c3-938a-3ff71b7a7c49 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:16.198441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 20/41] 10.222.0.58 () {58 vars in 1361 bytes} [Mon Oct 3 13:15:15 2022] PUT /identity/v3/domains/default/users/72593536090c47589cfad801ff3da440/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:15:16.198441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6f99b-9f61-4245-ad2c-aeb958a2c909 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:16.198441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6f99b-9f61-4245-ad2c-aeb958a2c909 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:16.198441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6f99b-9f61-4245-ad2c-aeb958a2c909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:16.236245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0be6f99b-9f61-4245-ad2c-aeb958a2c909 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:16.238684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0be6f99b-9f61-4245-ad2c-aeb958a2c909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:16.244599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 22/42] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:15:15 2022] POST /identity/v3/projects => generated 301 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:15:16.299211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:16.390064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603b41c-af63-4353-ac3e-98071db14c9f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:16.410040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 21/43] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:16.541809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:16.569418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603b41c-af63-4353-ac3e-98071db14c9f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:16.719535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a094c1d-69ec-4959-8b6c-31b37c955d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:16.720216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a094c1d-69ec-4959-8b6c-31b37c955d7a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:16.720593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a094c1d-69ec-4959-8b6c-31b37c955d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:16.720891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a094c1d-69ec-4959-8b6c-31b37c955d7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:16.722321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a094c1d-69ec-4959-8b6c-31b37c955d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:16.724075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a094c1d-69ec-4959-8b6c-31b37c955d7a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:16.726257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 23/44] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:16.732575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6f99b-9f61-4245-ad2c-aeb958a2c909 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:16.762953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a094c1d-69ec-4959-8b6c-31b37c955d7a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:16.923737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6f99b-9f61-4245-ad2c-aeb958a2c909 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:16.929357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a094c1d-69ec-4959-8b6c-31b37c955d7a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:16.985974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd8a1c-089a-4463-b18f-759facb097ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:16.986787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8a1c-089a-4463-b18f-759facb097ed None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:16.987215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8a1c-089a-4463-b18f-759facb097ed None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:16.987578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd8a1c-089a-4463-b18f-759facb097ed None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:16.988461 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd8a1c-089a-4463-b18f-759facb097ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:16.990985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd8a1c-089a-4463-b18f-759facb097ed None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:16.992092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4ebe4-6b6f-4834-ae99-d11ed888e063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:16.993471 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ebe4-6b6f-4834-ae99-d11ed888e063 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:16.994009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ebe4-6b6f-4834-ae99-d11ed888e063 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:16.994634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4ebe4-6b6f-4834-ae99-d11ed888e063 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:16.995831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4ebe4-6b6f-4834-ae99-d11ed888e063 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:16.998351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4ebe4-6b6f-4834-ae99-d11ed888e063 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:17.025287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 24/45] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Oct 3 13:15:16 2022] POST /identity/v3/roles => generated 223 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:15:17.041981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 22/46] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:15:16 2022] POST /identity/v3/projects => generated 312 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:15:18.725614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6c2e2-4d2b-4aef-948e-c25cf6f6f756 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:18.726426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6c2e2-4d2b-4aef-948e-c25cf6f6f756 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:18.726426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6c2e2-4d2b-4aef-948e-c25cf6f6f756 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:18.727548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 25/47] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:18.734421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb7f80-e566-4670-9207-11359a633b45 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:18.734921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb7f80-e566-4670-9207-11359a633b45 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:18.735536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feb7f80-e566-4670-9207-11359a633b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:18.766479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2feb7f80-e566-4670-9207-11359a633b45 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:18.768430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2feb7f80-e566-4670-9207-11359a633b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:18.775083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:18.853301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 23/48] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:19.135732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6c2e2-4d2b-4aef-948e-c25cf6f6f756 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:19.153782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:19.268639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6c2e2-4d2b-4aef-948e-c25cf6f6f756 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:19.309909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bd8ae-b5dc-4f5f-a76c-d7af5929061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:19.310719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bd8ae-b5dc-4f5f-a76c-d7af5929061c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:19.311094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bd8ae-b5dc-4f5f-a76c-d7af5929061c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:19.311461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bd8ae-b5dc-4f5f-a76c-d7af5929061c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:19.324991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bd8ae-b5dc-4f5f-a76c-d7af5929061c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6d3bd8ae-b5dc-4f5f-a76c-d7af5929061c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:19.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:19.332511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:19.333520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 26/49] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:15:19.342415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb7f80-e566-4670-9207-11359a633b45 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:19.438391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08013f1-aaa5-4b09-a1e3-0aebc895beac None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:19.438858 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08013f1-aaa5-4b09-a1e3-0aebc895beac None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:19.439322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08013f1-aaa5-4b09-a1e3-0aebc895beac None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:19.441403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 27/50] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:19.451087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e4939-6164-4d96-bcc2-c55564d41e74 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:19.451541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e4939-6164-4d96-bcc2-c55564d41e74 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:19.452000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e4939-6164-4d96-bcc2-c55564d41e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:19.472054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feb7f80-e566-4670-9207-11359a633b45 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:19.477532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1e9e4939-6164-4d96-bcc2-c55564d41e74 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:19.480265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1e9e4939-6164-4d96-bcc2-c55564d41e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:19.519644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:19.521399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:19.521895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:19.522275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:19.523416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:19.523883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:19.526462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:19.538134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 24/51] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:19.544233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:19.547370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:19.580318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 28/52] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:19.853804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e4939-6164-4d96-bcc2-c55564d41e74 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:19.856601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:19.861780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5f305-1e48-43ee-b75f-3755266ed049 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:19.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24893e69-16d1-4583-9f82-fe57e1963935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:19.913135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24893e69-16d1-4583-9f82-fe57e1963935 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:19.913621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24893e69-16d1-4583-9f82-fe57e1963935 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:19.914046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24893e69-16d1-4583-9f82-fe57e1963935 None admin] PATH_INFO: `/v3/users/72593536090c47589cfad801ff3da440` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:19.915339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24893e69-16d1-4583-9f82-fe57e1963935 None admin] RBAC: Authorizing `identity:get_user(user_id=72593536090c47589cfad801ff3da440)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:19.916866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24893e69-16d1-4583-9f82-fe57e1963935 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:19.918791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 25/53] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:19 2022] GET /identity/v3/users/72593536090c47589cfad801ff3da440 => generated 247 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:19.924376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24893e69-16d1-4583-9f82-fe57e1963935 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:20.000653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e4939-6164-4d96-bcc2-c55564d41e74 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:20.002994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:20.039452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba18c05-6a90-47c0-a25d-c7023613e284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:20.040168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba18c05-6a90-47c0-a25d-c7023613e284 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:20.040452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba18c05-6a90-47c0-a25d-c7023613e284 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:20.040733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba18c05-6a90-47c0-a25d-c7023613e284 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:20.042013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba18c05-6a90-47c0-a25d-c7023613e284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:20.043403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24893e69-16d1-4583-9f82-fe57e1963935 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:20.043544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba18c05-6a90-47c0-a25d-c7023613e284 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:20.045115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 29/54] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:20.050769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba18c05-6a90-47c0-a25d-c7023613e284 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:20.092208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616da105-5e51-4763-9f7c-5475970035e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:20.093007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da105-5e51-4763-9f7c-5475970035e0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:20.093375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da105-5e51-4763-9f7c-5475970035e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:20.093698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da105-5e51-4763-9f7c-5475970035e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:20.095013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616da105-5e51-4763-9f7c-5475970035e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:20.096578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616da105-5e51-4763-9f7c-5475970035e0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:20.098962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 26/55] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:20.105517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616da105-5e51-4763-9f7c-5475970035e0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:20.136178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:20.172285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba18c05-6a90-47c0-a25d-c7023613e284 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:20.211364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc302d-c94e-4f92-a00b-6b55baebc724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:20.212193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc302d-c94e-4f92-a00b-6b55baebc724 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:20.212562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc302d-c94e-4f92-a00b-6b55baebc724 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:20.212920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc302d-c94e-4f92-a00b-6b55baebc724 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:20.213720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc302d-c94e-4f92-a00b-6b55baebc724 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:20.215478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc302d-c94e-4f92-a00b-6b55baebc724 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:20.217788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616da105-5e51-4763-9f7c-5475970035e0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:20.245800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 30/56] 10.222.0.58 () {60 vars in 1140 bytes} [Mon Oct 3 13:15:20 2022] POST /identity/v3/projects => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:15:20.260315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b86a02-e3e5-4cd4-8eba-619788b309d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:20.260975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b86a02-e3e5-4cd4-8eba-619788b309d4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:20.261272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b86a02-e3e5-4cd4-8eba-619788b309d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:20.261587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b86a02-e3e5-4cd4-8eba-619788b309d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:20.262371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b86a02-e3e5-4cd4-8eba-619788b309d4 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.domain.id=default, user.id=72593536090c47589cfad801ff3da440) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:20.262693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b86a02-e3e5-4cd4-8eba-619788b309d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:20.264248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b86a02-e3e5-4cd4-8eba-619788b309d4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:20.272134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 27/57] 10.222.0.58 () {56 vars in 1322 bytes} [Mon Oct 3 13:15:20 2022] GET /identity/v3/role_assignments?user.id=72593536090c47589cfad801ff3da440&scope.domain.id=default&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 530 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:22.557837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a2ca-8d2e-4ad8-a9e0-328bd8285697 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:22.558693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a2ca-8d2e-4ad8-a9e0-328bd8285697 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:22.558858 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a2ca-8d2e-4ad8-a9e0-328bd8285697 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:22.560349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 31/58] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:22.566865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75400a0-6df5-4d4c-a659-941a96e2e824 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:22.567686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75400a0-6df5-4d4c-a659-941a96e2e824 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:22.567868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75400a0-6df5-4d4c-a659-941a96e2e824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:22.575219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:22.589822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e75400a0-6df5-4d4c-a659-941a96e2e824 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:22.590993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e75400a0-6df5-4d4c-a659-941a96e2e824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:22.661633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 28/59] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:22.945517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a2ca-8d2e-4ad8-a9e0-328bd8285697 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:23.060816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a2ca-8d2e-4ad8-a9e0-328bd8285697 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:23.086635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:23.099831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb96871-03d3-4b67-8013-182f14ff7ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:23.100480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb96871-03d3-4b67-8013-182f14ff7ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:23.100768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb96871-03d3-4b67-8013-182f14ff7ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:23.101052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb96871-03d3-4b67-8013-182f14ff7ac7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:23.102253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb96871-03d3-4b67-8013-182f14ff7ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:23.103727 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb96871-03d3-4b67-8013-182f14ff7ac7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:23.105335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 32/60] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:23.111055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75400a0-6df5-4d4c-a659-941a96e2e824 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:23.216192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75400a0-6df5-4d4c-a659-941a96e2e824 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:23.251601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:23.258101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbce5d5-301e-4a60-8634-db41f9dca1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:23.258754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbce5d5-301e-4a60-8634-db41f9dca1a5 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:23.259046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbce5d5-301e-4a60-8634-db41f9dca1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:23.259328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbce5d5-301e-4a60-8634-db41f9dca1a5 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:23.260033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbce5d5-301e-4a60-8634-db41f9dca1a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:23.261634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbce5d5-301e-4a60-8634-db41f9dca1a5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:23.293748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 29/61] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:15:23 2022] POST /identity/v3/users => generated 275 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:15:26.896127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed2a20-7197-4248-858d-c37b8a605da4 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:26.897242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed2a20-7197-4248-858d-c37b8a605da4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:26.897733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed2a20-7197-4248-858d-c37b8a605da4 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:26.901768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 33/62] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:26.915626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e60145-3ab6-43a6-b6da-5841381cdfe1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:26.916295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e60145-3ab6-43a6-b6da-5841381cdfe1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:26.916295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e60145-3ab6-43a6-b6da-5841381cdfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:26.944027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-74e60145-3ab6-43a6-b6da-5841381cdfe1 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:26.945294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74e60145-3ab6-43a6-b6da-5841381cdfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:27.065205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 30/63] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:27.573367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed2a20-7197-4248-858d-c37b8a605da4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:27.682906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddadf4-0507-4769-9938-a64649f51b00 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:27.683663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddadf4-0507-4769-9938-a64649f51b00 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:27.684525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddadf4-0507-4769-9938-a64649f51b00 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:27.687407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 31/64] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:27.696086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e976c-bb0b-4ef8-b753-0ec43e1dcb3c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:27.696688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e976c-bb0b-4ef8-b753-0ec43e1dcb3c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:27.697164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e976c-bb0b-4ef8-b753-0ec43e1dcb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:27.724671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c05e976c-bb0b-4ef8-b753-0ec43e1dcb3c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:27.727272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c05e976c-bb0b-4ef8-b753-0ec43e1dcb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:27.765399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed2a20-7197-4248-858d-c37b8a605da4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:27.843057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07740615-1bdf-4e43-b231-3d1c870921a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:27.844104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740615-1bdf-4e43-b231-3d1c870921a0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:27.844615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740615-1bdf-4e43-b231-3d1c870921a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:27.850584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07740615-1bdf-4e43-b231-3d1c870921a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:27.875582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07740615-1bdf-4e43-b231-3d1c870921a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-07740615-1bdf-4e43-b231-3d1c870921a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:27.882544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:27.883587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:27.884367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:27.891198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 32/65] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:27.892258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 34/66] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:15:27.902966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e976c-bb0b-4ef8-b753-0ec43e1dcb3c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:27.932732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5cc6-681c-4ca8-859d-c9051620a95d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:27.933297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5cc6-681c-4ca8-859d-c9051620a95d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:27.933297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5cc6-681c-4ca8-859d-c9051620a95d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:27.934782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 35/67] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:27.940866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc25d38-a39c-4513-a3ad-de2db8d6ecff None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:27.941394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc25d38-a39c-4513-a3ad-de2db8d6ecff None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:27.942514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc25d38-a39c-4513-a3ad-de2db8d6ecff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:27.971322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2fc25d38-a39c-4513-a3ad-de2db8d6ecff None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:27.974509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2fc25d38-a39c-4513-a3ad-de2db8d6ecff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:28.016661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:28.076075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:28.125489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e976c-bb0b-4ef8-b753-0ec43e1dcb3c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:28.126078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 36/68] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:28.128072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25374b51-027c-4513-b00f-383270ea90d3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.128711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25374b51-027c-4513-b00f-383270ea90d3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.129885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25374b51-027c-4513-b00f-383270ea90d3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:28.134135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 37/69] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:28.140720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410418b-8dd7-4515-98db-d82ce21dd314 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.141487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410418b-8dd7-4515-98db-d82ce21dd314 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.141991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410418b-8dd7-4515-98db-d82ce21dd314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:28.174762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8410418b-8dd7-4515-98db-d82ce21dd314 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:28.176944 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8410418b-8dd7-4515-98db-d82ce21dd314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:28.191039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:28.192595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.193141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.193715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:28.197218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:28.197218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:28.198292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:28.215241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 33/70] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:28.225647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:28.386557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 38/71] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:28.406154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ad049-013d-4565-ab2d-6636f09944d3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.406452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ad049-013d-4565-ab2d-6636f09944d3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.406737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ad049-013d-4565-ab2d-6636f09944d3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:28.408298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 39/72] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:28.418925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8c051-1d86-4ce0-85c8-a84582615eb6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.419408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8c051-1d86-4ce0-85c8-a84582615eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.419928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8c051-1d86-4ce0-85c8-a84582615eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:28.450266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-17b8c051-1d86-4ce0-85c8-a84582615eb6 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:28.453058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-17b8c051-1d86-4ce0-85c8-a84582615eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:28.496745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f2a14-fa87-4587-a032-95c03aba5bd4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:28.510116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:28.556110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145b639-cf2d-408a-a048-25b061383b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:28.557271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145b639-cf2d-408a-a048-25b061383b38 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.557887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145b639-cf2d-408a-a048-25b061383b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.558331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145b639-cf2d-408a-a048-25b061383b38 None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:28.561773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 40/73] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:28.563192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8c051-1d86-4ce0-85c8-a84582615eb6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:28.587175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145b639-cf2d-408a-a048-25b061383b38 None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:28.589095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145b639-cf2d-408a-a048-25b061383b38 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:28.592079 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 34/74] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:28 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:28.593538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145b639-cf2d-408a-a048-25b061383b38 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:28.617656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:28.703702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8c051-1d86-4ce0-85c8-a84582615eb6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:28.743094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145b639-cf2d-408a-a048-25b061383b38 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:28.745527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e87eb-b269-4765-956b-62d75014a134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:28.746226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e87eb-b269-4765-956b-62d75014a134 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.746589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e87eb-b269-4765-956b-62d75014a134 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.747778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e87eb-b269-4765-956b-62d75014a134 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:28.765992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e87eb-b269-4765-956b-62d75014a134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c09e87eb-b269-4765-956b-62d75014a134 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:28.773123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:28.774243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:28.775193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:28.778037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 41/75] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:28.779335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e87eb-b269-4765-956b-62d75014a134 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:28.790590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fd80b-b1e3-4295-b3b0-5414427b5578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:28.791469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd80b-b1e3-4295-b3b0-5414427b5578 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.791856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd80b-b1e3-4295-b3b0-5414427b5578 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.792269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd80b-b1e3-4295-b3b0-5414427b5578 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:28.806218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05fd80b-b1e3-4295-b3b0-5414427b5578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-a05fd80b-b1e3-4295-b3b0-5414427b5578 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:28.811767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:28.812871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:28.813889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:15:28.821083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 35/76] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:28.821083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fd80b-b1e3-4295-b3b0-5414427b5578 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:28.859994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:28.921833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:28.928720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e87eb-b269-4765-956b-62d75014a134 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:28.966632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fd80b-b1e3-4295-b3b0-5414427b5578 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:28.977416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f497f-6027-443d-9b2b-f126c6bf411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:28.978468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f497f-6027-443d-9b2b-f126c6bf411f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:28.978914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f497f-6027-443d-9b2b-f126c6bf411f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:28.979401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f497f-6027-443d-9b2b-f126c6bf411f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.019214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:29.020196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:29.020419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:29.020670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.021402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:29.021754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:29.023282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:29.040996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 36/77] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:29.040996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.072492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f497f-6027-443d-9b2b-f126c6bf411f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:29.074071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f497f-6027-443d-9b2b-f126c6bf411f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:29.169641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:29.181887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f121d34-d69d-49f8-be10-5ceb15ed4b82 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:29.201489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:29.213947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 42/78] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:29.215138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f497f-6027-443d-9b2b-f126c6bf411f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.221402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:29.222219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:29.222219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:29.223236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.223236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:29.223487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:29.224894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:29.234181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 37/79] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:29.235061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.359944 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f497f-6027-443d-9b2b-f126c6bf411f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:29.369156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce65f68-cacc-45a2-8b2b-10cc0260b9b2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:29.407255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a41c26-e5e3-4af8-808f-f5d4923c485a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:29.408119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a41c26-e5e3-4af8-808f-f5d4923c485a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:29.408544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a41c26-e5e3-4af8-808f-f5d4923c485a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:29.408924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a41c26-e5e3-4af8-808f-f5d4923c485a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.410462 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a41c26-e5e3-4af8-808f-f5d4923c485a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:29.411683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbc010-9d0a-43e4-846a-a48db1225132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:29.412393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbc010-9d0a-43e4-846a-a48db1225132 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:29.412548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a41c26-e5e3-4af8-808f-f5d4923c485a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:29.412688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbc010-9d0a-43e4-846a-a48db1225132 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:29.412951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becbc010-9d0a-43e4-846a-a48db1225132 None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.414395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbc010-9d0a-43e4-846a-a48db1225132 None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:29.414765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 43/80] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:29.415920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becbc010-9d0a-43e4-846a-a48db1225132 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:29.416198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a41c26-e5e3-4af8-808f-f5d4923c485a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.425352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 38/81] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:29 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:29.425352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbc010-9d0a-43e4-846a-a48db1225132 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.454864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:29.508720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:29.553419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a41c26-e5e3-4af8-808f-f5d4923c485a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:29.556327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becbc010-9d0a-43e4-846a-a48db1225132 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:29.596179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494c46a-8171-4102-8c2e-ecc8b74389ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:29.597043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c46a-8171-4102-8c2e-ecc8b74389ad None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:29.597413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c46a-8171-4102-8c2e-ecc8b74389ad None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:29.597853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c46a-8171-4102-8c2e-ecc8b74389ad None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.599401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494c46a-8171-4102-8c2e-ecc8b74389ad None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:29.601149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494c46a-8171-4102-8c2e-ecc8b74389ad None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:29.602914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 39/82] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:29 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:29.603739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494c46a-8171-4102-8c2e-ecc8b74389ad None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.616897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaeeef-edce-4470-83df-d3e686958d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:29.618021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaeeef-edce-4470-83df-d3e686958d7b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:29.618512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaeeef-edce-4470-83df-d3e686958d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:29.619000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eaeeef-edce-4470-83df-d3e686958d7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.638169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eaeeef-edce-4470-83df-d3e686958d7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c7eaeeef-edce-4470-83df-d3e686958d7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:29.647796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:29.648937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:29.650205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 44/83] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:15:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:29.651057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaeeef-edce-4470-83df-d3e686958d7b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.729866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:29.745320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494c46a-8171-4102-8c2e-ecc8b74389ad None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:29.786311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bd079-8d79-4b74-bbd2-fb2d4c258a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:29.786856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd079-8d79-4b74-bbd2-fb2d4c258a12 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:29.787147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd079-8d79-4b74-bbd2-fb2d4c258a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:29.787429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9bd079-8d79-4b74-bbd2-fb2d4c258a12 None admin] PATH_INFO: `/v3/users/72593536090c47589cfad801ff3da440` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.788656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bd079-8d79-4b74-bbd2-fb2d4c258a12 None admin] RBAC: Authorizing `identity:get_user(user_id=72593536090c47589cfad801ff3da440)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:29.790246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9bd079-8d79-4b74-bbd2-fb2d4c258a12 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:29.791929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 40/84] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:29 2022] GET /identity/v3/users/72593536090c47589cfad801ff3da440 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:29.792742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bd079-8d79-4b74-bbd2-fb2d4c258a12 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.811291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eaeeef-edce-4470-83df-d3e686958d7b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:29.842945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:29.852909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fdaa2-c38b-4599-b116-b8a5a2dd38a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:29.853588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fdaa2-c38b-4599-b116-b8a5a2dd38a2 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:29.853877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fdaa2-c38b-4599-b116-b8a5a2dd38a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:29.854192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fdaa2-c38b-4599-b116-b8a5a2dd38a2 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:29.854842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fdaa2-c38b-4599-b116-b8a5a2dd38a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:29.856355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42fdaa2-c38b-4599-b116-b8a5a2dd38a2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:29.887539 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 45/85] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:15:29 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:15:29.888444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fdaa2-c38b-4599-b116-b8a5a2dd38a2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:29.987444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9bd079-8d79-4b74-bbd2-fb2d4c258a12 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:30.029744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42fdaa2-c38b-4599-b116-b8a5a2dd38a2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:30.059013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176cd23-de73-4a43-8d88-dbb239903503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:30.060058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176cd23-de73-4a43-8d88-dbb239903503 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:30.060534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176cd23-de73-4a43-8d88-dbb239903503 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:30.061012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176cd23-de73-4a43-8d88-dbb239903503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:30.081867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:30.083423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:30.083892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:30.084370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:30.085745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] RBAC: Adding query filter params (role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=dda4727fec9747b4b20cb990f7305e3d, user.id=02a67abf7e2341ab9df44025a4aa90ee) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:30.086327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:30.088704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:30.103970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 46/86] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:29 2022] GET /identity/v3/role_assignments?user.id=02a67abf7e2341ab9df44025a4aa90ee&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:30.105383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:30.122979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:30.205230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:30.231983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176cd23-de73-4a43-8d88-dbb239903503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:30.231983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176cd23-de73-4a43-8d88-dbb239903503 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:30.290701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec9ac9-eeab-4839-a183-059fceea6b57 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:30.343175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:30.347290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:30.347290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:30.347290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:30.347290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:30.347290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:30.347290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:30.368298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 47/87] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:15:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:30.372540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:30.397031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 41/88] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:30.397794 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176cd23-de73-4a43-8d88-dbb239903503 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:30.494432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:30.561714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176cd23-de73-4a43-8d88-dbb239903503 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:30.575842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef65ea7-b294-4ce6-a689-6328ab236de2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:30.603174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:30.636428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda4d44-ff14-4a59-b393-b10e282342ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:30.637506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4d44-ff14-4a59-b393-b10e282342ca None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:30.638781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4d44-ff14-4a59-b393-b10e282342ca None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:30.639303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda4d44-ff14-4a59-b393-b10e282342ca None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:30.642900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda4d44-ff14-4a59-b393-b10e282342ca None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:30.646035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda4d44-ff14-4a59-b393-b10e282342ca None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:30.655858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 42/89] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:30 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:30.657093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda4d44-ff14-4a59-b393-b10e282342ca None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:30.665341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daa920-a948-42dd-9fd9-4c78e03d3ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:30.666378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa920-a948-42dd-9fd9-4c78e03d3ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:30.667723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa920-a948-42dd-9fd9-4c78e03d3ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:30.668215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa920-a948-42dd-9fd9-4c78e03d3ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:30.879613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daa920-a948-42dd-9fd9-4c78e03d3ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:30.882387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daa920-a948-42dd-9fd9-4c78e03d3ee0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:30.918872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda4d44-ff14-4a59-b393-b10e282342ca None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:30.982978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:30.996430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336b6df-638a-4bf7-902e-568fa4358011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:30.997102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336b6df-638a-4bf7-902e-568fa4358011 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:30.997387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336b6df-638a-4bf7-902e-568fa4358011 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:30.997697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336b6df-638a-4bf7-902e-568fa4358011 None admin] PATH_INFO: `/v3/projects/9c725885325642f88ad12b0f1492310d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:30.998888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336b6df-638a-4bf7-902e-568fa4358011 None admin] RBAC: Authorizing `identity:get_project(project_id=9c725885325642f88ad12b0f1492310d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:31.000381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336b6df-638a-4bf7-902e-568fa4358011 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:31.002474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 43/90] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:30 2022] GET /identity/v3/projects/9c725885325642f88ad12b0f1492310d => generated 312 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:31.003304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336b6df-638a-4bf7-902e-568fa4358011 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:31.086663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 48/91] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:31.087557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daa920-a948-42dd-9fd9-4c78e03d3ee0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:31.100340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:31.208056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daa920-a948-42dd-9fd9-4c78e03d3ee0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:31.250119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900441ae-1d13-484f-8251-86804978e63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:31.250756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900441ae-1d13-484f-8251-86804978e63e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:31.251045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900441ae-1d13-484f-8251-86804978e63e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:31.251330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900441ae-1d13-484f-8251-86804978e63e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:31.252078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900441ae-1d13-484f-8251-86804978e63e None admin] RBAC: Adding query filter params (role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=9c725885325642f88ad12b0f1492310d, user.id=02a67abf7e2341ab9df44025a4aa90ee) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:31.252396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900441ae-1d13-484f-8251-86804978e63e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:31.254470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900441ae-1d13-484f-8251-86804978e63e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:31.262220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 49/92] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:31 2022] GET /identity/v3/role_assignments?user.id=02a67abf7e2341ab9df44025a4aa90ee&scope.project.id=9c725885325642f88ad12b0f1492310d&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:31.263185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900441ae-1d13-484f-8251-86804978e63e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:31.333538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336b6df-638a-4bf7-902e-568fa4358011 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:31.431450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2bbf4-2505-4504-b00e-0250f957a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:31.432126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2bbf4-2505-4504-b00e-0250f957a956 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:31.432416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2bbf4-2505-4504-b00e-0250f957a956 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:31.432700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2bbf4-2505-4504-b00e-0250f957a956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:31.453045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:31.489580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900441ae-1d13-484f-8251-86804978e63e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:31.531173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ea484-58c9-45c9-9b10-525337f19664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:31.531845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ea484-58c9-45c9-9b10-525337f19664 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:31.532138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ea484-58c9-45c9-9b10-525337f19664 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:31.532419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ea484-58c9-45c9-9b10-525337f19664 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:31.534439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ea484-58c9-45c9-9b10-525337f19664 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:31.536540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ea484-58c9-45c9-9b10-525337f19664 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:31.538478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 50/93] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:31 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:31.550539 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ea484-58c9-45c9-9b10-525337f19664 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:31.679093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:31.681575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ea484-58c9-45c9-9b10-525337f19664 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:31.722356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2bbf4-2505-4504-b00e-0250f957a956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:31.724841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2bbf4-2505-4504-b00e-0250f957a956 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:31.724998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:31.725604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:31.725926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:31.726171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:31.727277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=dda4727fec9747b4b20cb990f7305e3d, user.id=72593536090c47589cfad801ff3da440) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:31.727600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:31.729140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:31.736458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 51/94] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:31 2022] GET /identity/v3/role_assignments?user.id=72593536090c47589cfad801ff3da440&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:31.904875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 44/95] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 902 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:31.911505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:31.927764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:32.044221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3b7cd-29ca-47e7-97d4-5a93fc81f740 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:32.081529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0ed7c-b7c0-45e0-9a93-9925d8b2b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:32.082236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0ed7c-b7c0-45e0-9a93-9925d8b2b594 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:32.082530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0ed7c-b7c0-45e0-9a93-9925d8b2b594 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:32.082815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0ed7c-b7c0-45e0-9a93-9925d8b2b594 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:32.084250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0ed7c-b7c0-45e0-9a93-9925d8b2b594 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:32.085812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0ed7c-b7c0-45e0-9a93-9925d8b2b594 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:32.087593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 52/96] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:31 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:32.097133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2bbf4-2505-4504-b00e-0250f957a956 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:32.233431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:32.245372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2bbf4-2505-4504-b00e-0250f957a956 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:32.293319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362eba7-12c7-4489-b5c6-6b55d8e4d4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:32.293991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eba7-12c7-4489-b5c6-6b55d8e4d4ba None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:32.294279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eba7-12c7-4489-b5c6-6b55d8e4d4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:32.294558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eba7-12c7-4489-b5c6-6b55d8e4d4ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:32.295316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362eba7-12c7-4489-b5c6-6b55d8e4d4ba None admin] RBAC: Adding query filter params (role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=dda4727fec9747b4b20cb990f7305e3d, user.id=02a67abf7e2341ab9df44025a4aa90ee) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:32.295633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362eba7-12c7-4489-b5c6-6b55d8e4d4ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:32.297131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362eba7-12c7-4489-b5c6-6b55d8e4d4ba None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:32.303933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 45/97] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:32 2022] GET /identity/v3/role_assignments?user.id=02a67abf7e2341ab9df44025a4aa90ee&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:33.563642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1366d-3a81-4a36-bb01-923f26c1109c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:33.564237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1366d-3a81-4a36-bb01-923f26c1109c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:33.564237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1366d-3a81-4a36-bb01-923f26c1109c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:33.566021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 53/98] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:33.572779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d3c0-6f00-49ec-b8c0-eb7fae6ad1d5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:33.573312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d3c0-6f00-49ec-b8c0-eb7fae6ad1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:33.573312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d3c0-6f00-49ec-b8c0-eb7fae6ad1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:33.598138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1945d3c0-6f00-49ec-b8c0-eb7fae6ad1d5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:33.599294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1945d3c0-6f00-49ec-b8c0-eb7fae6ad1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:33.629946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66760-d88e-4b15-875a-fb315a8877ff None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:33.630657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66760-d88e-4b15-875a-fb315a8877ff None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:33.631155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c66760-d88e-4b15-875a-fb315a8877ff None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:33.633075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 54/99] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:33.639051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101075-a3fc-43e5-946a-ee14efa4effb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:33.639730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101075-a3fc-43e5-946a-ee14efa4effb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:33.640226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101075-a3fc-43e5-946a-ee14efa4effb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:33.670576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c8101075-a3fc-43e5-946a-ee14efa4effb None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:33.671834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c8101075-a3fc-43e5-946a-ee14efa4effb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:33.727973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 46/100] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:33.752647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6735b5-a10f-42ec-8216-019cac4dc994 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:33.753162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6735b5-a10f-42ec-8216-019cac4dc994 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:33.753727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6735b5-a10f-42ec-8216-019cac4dc994 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:33.757164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 47/101] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:33.763947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01877525-7cb6-41d9-8b6b-79b25076982e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:33.764476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01877525-7cb6-41d9-8b6b-79b25076982e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:33.764990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01877525-7cb6-41d9-8b6b-79b25076982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:33.792355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 55/102] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:33.827162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-01877525-7cb6-41d9-8b6b-79b25076982e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:33.829020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-01877525-7cb6-41d9-8b6b-79b25076982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:34.005901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 48/103] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:34.144559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8101075-a3fc-43e5-946a-ee14efa4effb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:34.148710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:34.239010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01877525-7cb6-41d9-8b6b-79b25076982e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:34.280765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:34.309756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8101075-a3fc-43e5-946a-ee14efa4effb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:34.375352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ce7db-77e9-4334-9d42-f9b2fba0cc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:34.376475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce7db-77e9-4334-9d42-f9b2fba0cc1d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:34.376993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce7db-77e9-4334-9d42-f9b2fba0cc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:34.377531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ce7db-77e9-4334-9d42-f9b2fba0cc1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:34.379688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ce7db-77e9-4334-9d42-f9b2fba0cc1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:34.382849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ce7db-77e9-4334-9d42-f9b2fba0cc1d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:34.386122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 56/104] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:34.395296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ce7db-77e9-4334-9d42-f9b2fba0cc1d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:34.513829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01877525-7cb6-41d9-8b6b-79b25076982e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:34.561153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ce7db-77e9-4334-9d42-f9b2fba0cc1d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:34.563443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:34.604658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d4999-435b-4375-bab0-98791492b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:34.605569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d4999-435b-4375-bab0-98791492b183 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:34.605691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d4999-435b-4375-bab0-98791492b183 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:34.606491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d4999-435b-4375-bab0-98791492b183 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:34.621932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d4999-435b-4375-bab0-98791492b183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d86d4999-435b-4375-bab0-98791492b183 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:34.636707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:34.637782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 49/105] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:15:34.639378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d4999-435b-4375-bab0-98791492b183 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:34.670327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77928614-cf1b-491e-8283-7e3b17494273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:34.671040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77928614-cf1b-491e-8283-7e3b17494273 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:34.671292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77928614-cf1b-491e-8283-7e3b17494273 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:34.671576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77928614-cf1b-491e-8283-7e3b17494273 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:34.672280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77928614-cf1b-491e-8283-7e3b17494273 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:34.673884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77928614-cf1b-491e-8283-7e3b17494273 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:34.721892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 57/106] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:15:34 2022] POST /identity/v3/projects => generated 302 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:15:34.723560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77928614-cf1b-491e-8283-7e3b17494273 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:34.811117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:34.872487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d4999-435b-4375-bab0-98791492b183 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:34.976373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff08a-8a8d-4689-aaee-df388c8c20a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:34.977057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff08a-8a8d-4689-aaee-df388c8c20a2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:34.977345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff08a-8a8d-4689-aaee-df388c8c20a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:34.977659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff08a-8a8d-4689-aaee-df388c8c20a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:34.994485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77928614-cf1b-491e-8283-7e3b17494273 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:35.010259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff08a-8a8d-4689-aaee-df388c8c20a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-a48ff08a-8a8d-4689-aaee-df388c8c20a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:35.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:35.024670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:15:35.025481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 50/107] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:35.032930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff08a-8a8d-4689-aaee-df388c8c20a2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:35.077304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:35.114264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73380a9a-9c8a-46e5-9bde-c97ebbeba9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:35.115190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380a9a-9c8a-46e5-9bde-c97ebbeba9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:35.115487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380a9a-9c8a-46e5-9bde-c97ebbeba9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:35.115775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380a9a-9c8a-46e5-9bde-c97ebbeba9e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:35.116488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73380a9a-9c8a-46e5-9bde-c97ebbeba9e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:35.116805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73380a9a-9c8a-46e5-9bde-c97ebbeba9e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:35.118305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73380a9a-9c8a-46e5-9bde-c97ebbeba9e7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:35.144506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 58/108] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:35.145710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9049-86bd-4baa-a9c0-f2d98d4163ee None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:35.145710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9049-86bd-4baa-a9c0-f2d98d4163ee None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:35.145710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9049-86bd-4baa-a9c0-f2d98d4163ee None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:35.147583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 59/109] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:35.153443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b9049-86bd-4baa-a9c0-f2d98d4163ee None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:35.322743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff08a-8a8d-4689-aaee-df388c8c20a2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:35.386885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b9049-86bd-4baa-a9c0-f2d98d4163ee None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:35.407030 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e4957-7e8e-45d9-a4e1-607e4190e743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:35.407657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e4957-7e8e-45d9-a4e1-607e4190e743 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:35.407906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e4957-7e8e-45d9-a4e1-607e4190e743 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:35.408135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e4957-7e8e-45d9-a4e1-607e4190e743 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:35.408857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e4957-7e8e-45d9-a4e1-607e4190e743 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:35.409175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e4957-7e8e-45d9-a4e1-607e4190e743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:35.411909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e4957-7e8e-45d9-a4e1-607e4190e743 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:35.418053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:35.430166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 51/110] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:35.435134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84769c-dafa-447d-ba5f-ca83c5dbf49e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:35.435422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84769c-dafa-447d-ba5f-ca83c5dbf49e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:35.435773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b84769c-dafa-447d-ba5f-ca83c5dbf49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:35.470370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb7e9a-e155-453f-80e9-ee57b212336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:35.471042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb7e9a-e155-453f-80e9-ee57b212336d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:35.471334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb7e9a-e155-453f-80e9-ee57b212336d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:35.471623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb7e9a-e155-453f-80e9-ee57b212336d None admin] PATH_INFO: `/v3/users/32f3633da25b4bea87035e4ebb60a78d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:35.474075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9b84769c-dafa-447d-ba5f-ca83c5dbf49e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:35.475337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9b84769c-dafa-447d-ba5f-ca83c5dbf49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:35.503254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb7e9a-e155-453f-80e9-ee57b212336d None admin] RBAC: Authorizing `identity:get_user(user_id=32f3633da25b4bea87035e4ebb60a78d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:35.504892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb7e9a-e155-453f-80e9-ee57b212336d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:35.507444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 60/111] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:35 2022] GET /identity/v3/users/32f3633da25b4bea87035e4ebb60a78d => generated 289 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:35.508846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb7e9a-e155-453f-80e9-ee57b212336d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:35.615454 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:35.623997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 52/112] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:35.627253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84769c-dafa-447d-ba5f-ca83c5dbf49e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:35.780719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb7e9a-e155-453f-80e9-ee57b212336d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:35.844283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b84769c-dafa-447d-ba5f-ca83c5dbf49e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:35.859050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bce87-3dd0-4c90-82fe-5011a2c701e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:35.860363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bce87-3dd0-4c90-82fe-5011a2c701e1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:35.861007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bce87-3dd0-4c90-82fe-5011a2c701e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:35.861594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4bce87-3dd0-4c90-82fe-5011a2c701e1 None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:35.864380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bce87-3dd0-4c90-82fe-5011a2c701e1 None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:35.867271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4bce87-3dd0-4c90-82fe-5011a2c701e1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:35.871870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 61/113] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:35 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:35.882575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bce87-3dd0-4c90-82fe-5011a2c701e1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:35.883098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:35.904350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ab19f-8aaa-41dd-85f0-0668540c85e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:35.905305 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab19f-8aaa-41dd-85f0-0668540c85e5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:35.905777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab19f-8aaa-41dd-85f0-0668540c85e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:35.906314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ab19f-8aaa-41dd-85f0-0668540c85e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:36.021393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ab19f-8aaa-41dd-85f0-0668540c85e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:36.023213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ab19f-8aaa-41dd-85f0-0668540c85e5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:36.074574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4bce87-3dd0-4c90-82fe-5011a2c701e1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:36.101105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:36.142287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f411861-c5b7-4cfb-b051-ed8ede5b10d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:36.143947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f411861-c5b7-4cfb-b051-ed8ede5b10d8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:36.145199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f411861-c5b7-4cfb-b051-ed8ede5b10d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:36.145830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f411861-c5b7-4cfb-b051-ed8ede5b10d8 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:36.148002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 53/114] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:36.148330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f411861-c5b7-4cfb-b051-ed8ede5b10d8 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:36.153497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f411861-c5b7-4cfb-b051-ed8ede5b10d8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:36.154050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a750-7bcb-4e85-8b59-c1b1ed44931c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:36.154538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a750-7bcb-4e85-8b59-c1b1ed44931c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:36.155053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99a750-7bcb-4e85-8b59-c1b1ed44931c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:36.162431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 62/116] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:35 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:36.162815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 54/116] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:36.165264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f411861-c5b7-4cfb-b051-ed8ede5b10d8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:36.167178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0a548-76e0-4b65-bd35-9b437e428bfd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:36.167718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0a548-76e0-4b65-bd35-9b437e428bfd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:36.169099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0a548-76e0-4b65-bd35-9b437e428bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:36.208776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a4f0a548-76e0-4b65-bd35-9b437e428bfd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:36.210432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a4f0a548-76e0-4b65-bd35-9b437e428bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:36.288388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:36.387109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f411861-c5b7-4cfb-b051-ed8ede5b10d8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:36.417346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 55/117] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:36.419410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0a548-76e0-4b65-bd35-9b437e428bfd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:36.451351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef35f5-7a1c-4805-b5d2-22aa6a949998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:36.452013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef35f5-7a1c-4805-b5d2-22aa6a949998 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:36.452303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef35f5-7a1c-4805-b5d2-22aa6a949998 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:36.452585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef35f5-7a1c-4805-b5d2-22aa6a949998 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:36.454738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef35f5-7a1c-4805-b5d2-22aa6a949998 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:36.461090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef35f5-7a1c-4805-b5d2-22aa6a949998 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:36.462965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 63/118] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:36 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:36.463777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef35f5-7a1c-4805-b5d2-22aa6a949998 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:36.584004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:36.680894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0a548-76e0-4b65-bd35-9b437e428bfd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:36.702242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef35f5-7a1c-4805-b5d2-22aa6a949998 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:36.779202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d2586-3201-4a1b-84bf-6ba9253bc9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:36.782776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d2586-3201-4a1b-84bf-6ba9253bc9bb None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:36.786272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d2586-3201-4a1b-84bf-6ba9253bc9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:36.789366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d2586-3201-4a1b-84bf-6ba9253bc9bb None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d/users/02a67abf7e2341ab9df44025a4aa90ee/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:36.795248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d2586-3201-4a1b-84bf-6ba9253bc9bb None admin] RBAC: Authorizing `identity:create_grant(project_id=dda4727fec9747b4b20cb990f7305e3d, user_id=02a67abf7e2341ab9df44025a4aa90ee, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:36.799097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d2586-3201-4a1b-84bf-6ba9253bc9bb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:36.820680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230ffc1-39aa-40bb-b6d5-ce265152bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:36.822525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230ffc1-39aa-40bb-b6d5-ce265152bf7f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:36.823636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230ffc1-39aa-40bb-b6d5-ce265152bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:36.824574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230ffc1-39aa-40bb-b6d5-ce265152bf7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:36.837802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 56/119] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:15:36 2022] PUT /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d/users/02a67abf7e2341ab9df44025a4aa90ee/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:15:36.838667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d2586-3201-4a1b-84bf-6ba9253bc9bb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:36.853694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f230ffc1-39aa-40bb-b6d5-ce265152bf7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:36.867240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f230ffc1-39aa-40bb-b6d5-ce265152bf7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:36.868624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:36.870575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:36.871689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:36.876179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 64/120] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:15:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:36.880941 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230ffc1-39aa-40bb-b6d5-ce265152bf7f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:37.081056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d2586-3201-4a1b-84bf-6ba9253bc9bb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:37.163345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f230ffc1-39aa-40bb-b6d5-ce265152bf7f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:37.173768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:37.175340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:37.176727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:37.176900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:37.177629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:37.179395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] RBAC: Adding query filter params (role.id=cd3aa49828434199911a5124c59be2a4, scope.project.id=dda4727fec9747b4b20cb990f7305e3d, user.id=32f3633da25b4bea87035e4ebb60a78d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:37.180207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:37.183500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:37.192500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 57/121] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:36 2022] GET /identity/v3/role_assignments?user.id=32f3633da25b4bea87035e4ebb60a78d&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=cd3aa49828434199911a5124c59be2a4 => generated 259 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:37.193826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:37.248861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e08b8-592b-4870-988f-24c60041b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:37.249552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e08b8-592b-4870-988f-24c60041b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:37.249843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e08b8-592b-4870-988f-24c60041b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:37.250129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e08b8-592b-4870-988f-24c60041b7f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:37.280187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e08b8-592b-4870-988f-24c60041b7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4d9e08b8-592b-4870-988f-24c60041b7f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:37.285354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:37.286302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:37.287166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 65/122] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:37.288351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29de309-7527-4c8f-b027-8997321b7858 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:37.288351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29de309-7527-4c8f-b027-8997321b7858 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:37.288521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29de309-7527-4c8f-b027-8997321b7858 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:37.290940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 66/123] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:37.300134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aef4e-1d38-493c-911e-d4c734312e9a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:37.300696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aef4e-1d38-493c-911e-d4c734312e9a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:37.301210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aef4e-1d38-493c-911e-d4c734312e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:37.348498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6d9aef4e-1d38-493c-911e-d4c734312e9a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:37.348498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6d9aef4e-1d38-493c-911e-d4c734312e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:37.394358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:37.440724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849e012-01d1-46c4-a52a-11d3d1e97475 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:37.524536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 67/124] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:37.525505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9aef4e-1d38-493c-911e-d4c734312e9a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:37.532527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:37.533185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:37.533652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:37.533949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:37.534669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:37.535323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:37.538222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:37.548415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 58/125] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:37.555357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:37.760933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd80baa-fa5f-48bf-ada6-6150309b8b23 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:37.840848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce7177-044f-4e71-98c2-62bac5e2b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:37.841966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce7177-044f-4e71-98c2-62bac5e2b848 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:37.842475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce7177-044f-4e71-98c2-62bac5e2b848 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:37.842978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce7177-044f-4e71-98c2-62bac5e2b848 None admin] PATH_INFO: `/v3/users/72593536090c47589cfad801ff3da440` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:37.848452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce7177-044f-4e71-98c2-62bac5e2b848 None admin] RBAC: Authorizing `identity:get_user(user_id=72593536090c47589cfad801ff3da440)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:37.851467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce7177-044f-4e71-98c2-62bac5e2b848 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:37.854693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 59/126] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:37 2022] GET /identity/v3/users/72593536090c47589cfad801ff3da440 => generated 247 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:37.862915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce7177-044f-4e71-98c2-62bac5e2b848 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:37.880414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:37.895144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9aef4e-1d38-493c-911e-d4c734312e9a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:37.926602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:38.044402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:38.045294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:38.045804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:38.046254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:38.056510 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:38.057094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:38.059704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:38.109556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 68/127] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 584 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:38.137079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:38.390663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d1e13-e81d-4b9c-b2b9-2092a31f677d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:38.421788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce7177-044f-4e71-98c2-62bac5e2b848 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:38.514332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cf59c-f659-40a0-9745-e8d7aefce75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:38.516176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cf59c-f659-40a0-9745-e8d7aefce75d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:38.516928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cf59c-f659-40a0-9745-e8d7aefce75d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:38.517947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cf59c-f659-40a0-9745-e8d7aefce75d None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:38.522261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cf59c-f659-40a0-9745-e8d7aefce75d None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:38.525377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cf59c-f659-40a0-9745-e8d7aefce75d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:38.529121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 69/128] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:38 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:38.538445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cf59c-f659-40a0-9745-e8d7aefce75d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:38.566954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0954b5-e4cd-4716-98cb-f246582b6b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:38.568658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0954b5-e4cd-4716-98cb-f246582b6b83 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:38.569364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0954b5-e4cd-4716-98cb-f246582b6b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:38.570087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0954b5-e4cd-4716-98cb-f246582b6b83 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:38.572321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0954b5-e4cd-4716-98cb-f246582b6b83 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:38.575208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0954b5-e4cd-4716-98cb-f246582b6b83 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:38.582990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 60/129] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:37 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 720 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:38.585691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1d2ac-5e48-45c7-83c5-e6566cb84201 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:38.589638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1d2ac-5e48-45c7-83c5-e6566cb84201 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:38.593273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1d2ac-5e48-45c7-83c5-e6566cb84201 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:38.601978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 61/130] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:38 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:38.603791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1d2ac-5e48-45c7-83c5-e6566cb84201 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:38.656157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:38.836521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:38.882624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cf59c-f659-40a0-9745-e8d7aefce75d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:38.883376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1d2ac-5e48-45c7-83c5-e6566cb84201 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:38.945883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881e5be-b7c6-40c4-b70b-a0e1d5bf8827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:38.946554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881e5be-b7c6-40c4-b70b-a0e1d5bf8827 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:38.946849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881e5be-b7c6-40c4-b70b-a0e1d5bf8827 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:38.947130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881e5be-b7c6-40c4-b70b-a0e1d5bf8827 None admin] PATH_INFO: `/v3/projects/9c725885325642f88ad12b0f1492310d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:38.948314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881e5be-b7c6-40c4-b70b-a0e1d5bf8827 None admin] RBAC: Authorizing `identity:get_project(project_id=9c725885325642f88ad12b0f1492310d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:38.949861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881e5be-b7c6-40c4-b70b-a0e1d5bf8827 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:38.951562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 62/131] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:38 2022] GET /identity/v3/projects/9c725885325642f88ad12b0f1492310d => generated 312 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:38.952349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881e5be-b7c6-40c4-b70b-a0e1d5bf8827 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:38.958105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a56b1-a764-49f0-a8d5-4568188d1939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:38.959665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a56b1-a764-49f0-a8d5-4568188d1939 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:38.960481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a56b1-a764-49f0-a8d5-4568188d1939 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:38.961244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a56b1-a764-49f0-a8d5-4568188d1939 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:38.983367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a56b1-a764-49f0-a8d5-4568188d1939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-556a56b1-a764-49f0-a8d5-4568188d1939 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:39.001715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:39.003342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:39.004734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:39.009100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 70/132] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:15:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 472 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:39.011735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b904da-897a-4da6-8631-4eb163810a00 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:39.012534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b904da-897a-4da6-8631-4eb163810a00 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:39.013296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b904da-897a-4da6-8631-4eb163810a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:39.045625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c0b904da-897a-4da6-8631-4eb163810a00 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:39.048156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c0b904da-897a-4da6-8631-4eb163810a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:39.071828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881e5be-b7c6-40c4-b70b-a0e1d5bf8827 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:39.110061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523d095-0f7d-4a95-922b-98edb9ac3300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:39.110714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d095-0f7d-4a95-922b-98edb9ac3300 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:39.111003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d095-0f7d-4a95-922b-98edb9ac3300 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:39.111286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523d095-0f7d-4a95-922b-98edb9ac3300 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d/users/72593536090c47589cfad801ff3da440/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:39.113403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523d095-0f7d-4a95-922b-98edb9ac3300 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda4727fec9747b4b20cb990f7305e3d, user_id=72593536090c47589cfad801ff3da440, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:39.115038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f523d095-0f7d-4a95-922b-98edb9ac3300 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:39.142085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 63/133] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:15:38 2022] PUT /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d/users/72593536090c47589cfad801ff3da440/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:15:39.143593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523d095-0f7d-4a95-922b-98edb9ac3300 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:39.173858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:39.206039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 71/134] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:39.209348 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b904da-897a-4da6-8631-4eb163810a00 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:39.298145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523d095-0f7d-4a95-922b-98edb9ac3300 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:39.301188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:39.407311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c334b-112c-491d-a4bf-613bb78e2931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:39.408674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c334b-112c-491d-a4bf-613bb78e2931 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:39.409025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c334b-112c-491d-a4bf-613bb78e2931 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:39.413483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c334b-112c-491d-a4bf-613bb78e2931 None admin] PATH_INFO: `/v3/projects/9c725885325642f88ad12b0f1492310d/users/02a67abf7e2341ab9df44025a4aa90ee/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:39.421490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c334b-112c-491d-a4bf-613bb78e2931 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c725885325642f88ad12b0f1492310d, user_id=02a67abf7e2341ab9df44025a4aa90ee, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:39.421490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c334b-112c-491d-a4bf-613bb78e2931 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:39.465598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 64/135] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:15:39 2022] PUT /identity/v3/projects/9c725885325642f88ad12b0f1492310d/users/02a67abf7e2341ab9df44025a4aa90ee/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:15:39.573618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b904da-897a-4da6-8631-4eb163810a00 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:39.623153 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c334b-112c-491d-a4bf-613bb78e2931 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:39.702426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:39.704010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:39.707468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:39.708584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:39.710085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:39.710860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:39.722200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:39.743861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 72/136] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:15:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 537 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:39.761206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:39.842033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:39.953172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c334b-112c-491d-a4bf-613bb78e2931 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:40.055010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:40.060521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef560c0b-4c53-4ec9-aefc-0ae4830db93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:40.066010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef560c0b-4c53-4ec9-aefc-0ae4830db93a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:40.066010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef560c0b-4c53-4ec9-aefc-0ae4830db93a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:40.066010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef560c0b-4c53-4ec9-aefc-0ae4830db93a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:40.073010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef560c0b-4c53-4ec9-aefc-0ae4830db93a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:40.075848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef560c0b-4c53-4ec9-aefc-0ae4830db93a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:40.083894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 65/137] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:40.106062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f887c-a6bc-41e1-8758-4048158c98a1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:40.112790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef560c0b-4c53-4ec9-aefc-0ae4830db93a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:40.233763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a64b68-709e-43a0-89a4-6a613857228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:40.234755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a64b68-709e-43a0-89a4-6a613857228d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:40.235253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a64b68-709e-43a0-89a4-6a613857228d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:40.235750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a64b68-709e-43a0-89a4-6a613857228d None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:40.242211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a64b68-709e-43a0-89a4-6a613857228d None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:40.245161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a64b68-709e-43a0-89a4-6a613857228d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:40.251775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 73/138] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:39 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 491 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:40.261404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a64b68-709e-43a0-89a4-6a613857228d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:40.350295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef560c0b-4c53-4ec9-aefc-0ae4830db93a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:40.429496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a53dbe-d236-4b06-82bc-9821231f65ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:40.429496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a53dbe-d236-4b06-82bc-9821231f65ab None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:40.429496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a53dbe-d236-4b06-82bc-9821231f65ab None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:40.429496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a53dbe-d236-4b06-82bc-9821231f65ab None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:40.430110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a53dbe-d236-4b06-82bc-9821231f65ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:40.431585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a53dbe-d236-4b06-82bc-9821231f65ab None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:40.451519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:40.477734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 66/139] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:15:40 2022] POST /identity/v3/users => generated 283 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:15:40.486021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9dc6-fa4c-4f41-be18-78a9da4ff67d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:40.486618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9dc6-fa4c-4f41-be18-78a9da4ff67d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:40.487115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca9dc6-fa4c-4f41-be18-78a9da4ff67d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:40.490572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 67/140] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:40 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:40.500569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206fc41-98ef-4827-88dd-93715fedad28 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:40.501060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206fc41-98ef-4827-88dd-93715fedad28 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:40.501570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206fc41-98ef-4827-88dd-93715fedad28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:40.557688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0206fc41-98ef-4827-88dd-93715fedad28 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:40.560195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0206fc41-98ef-4827-88dd-93715fedad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:40.749658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a64b68-709e-43a0-89a4-6a613857228d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:40.801564 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 68/141] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:40.866006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815096f-60a0-49e0-9eb5-171df838688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:40.867115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815096f-60a0-49e0-9eb5-171df838688a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:40.867610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815096f-60a0-49e0-9eb5-171df838688a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:40.868104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815096f-60a0-49e0-9eb5-171df838688a None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:40.875024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815096f-60a0-49e0-9eb5-171df838688a None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:40.877883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815096f-60a0-49e0-9eb5-171df838688a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:40.881759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 74/142] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:40 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 621 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:40.891644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206fc41-98ef-4827-88dd-93715fedad28 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:41.181264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206fc41-98ef-4827-88dd-93715fedad28 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:41.231474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815096f-60a0-49e0-9eb5-171df838688a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:41.237257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:41.277022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea7817-0729-4ab9-b66b-e25ef4b197f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:41.278535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea7817-0729-4ab9-b66b-e25ef4b197f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:41.279308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea7817-0729-4ab9-b66b-e25ef4b197f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:41.280029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea7817-0729-4ab9-b66b-e25ef4b197f8 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d/users/02a67abf7e2341ab9df44025a4aa90ee/roles/753d3bbfcdaa4cef86cc8de37ae19906` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:41.290469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea7817-0729-4ab9-b66b-e25ef4b197f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda4727fec9747b4b20cb990f7305e3d, user_id=02a67abf7e2341ab9df44025a4aa90ee, role_id=753d3bbfcdaa4cef86cc8de37ae19906)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:41.293082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:41.297371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea7817-0729-4ab9-b66b-e25ef4b197f8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:41.330923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 69/143] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:15:40 2022] PUT /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d/users/02a67abf7e2341ab9df44025a4aa90ee/roles/753d3bbfcdaa4cef86cc8de37ae19906 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:15:41.518267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815096f-60a0-49e0-9eb5-171df838688a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:41.602337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584d735-c623-4280-aa6d-f344d048c738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:41.603410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d735-c623-4280-aa6d-f344d048c738 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:41.603905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d735-c623-4280-aa6d-f344d048c738 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:41.604363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d735-c623-4280-aa6d-f344d048c738 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:41.630543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584d735-c623-4280-aa6d-f344d048c738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f584d735-c623-4280-aa6d-f344d048c738 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:41.642480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:41.644206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:41.646285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 75/144] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:15:41.652158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea7817-0729-4ab9-b66b-e25ef4b197f8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:41.794569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92fca-3216-45f5-90a4-e0b6ee330a57 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:41.794973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92fca-3216-45f5-90a4-e0b6ee330a57 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:41.795367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d92fca-3216-45f5-90a4-e0b6ee330a57 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:41.797644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 76/145] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:41.806274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d4f3-4f05-4eb7-b583-c77d97978e69 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:41.806680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d4f3-4f05-4eb7-b583-c77d97978e69 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:41.807138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9d4f3-4f05-4eb7-b583-c77d97978e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:41.849262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d5e9d4f3-4f05-4eb7-b583-c77d97978e69 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:41.851384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d5e9d4f3-4f05-4eb7-b583-c77d97978e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:41.944975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea7817-0729-4ab9-b66b-e25ef4b197f8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:42.043588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:42.044240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:42.044528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:42.044901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:42.045527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:42.045843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:42.047406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:42.064089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 70/146] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:42.072060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:42.073029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 77/147] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:42.177788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:42.310134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b4d77-c793-4f31-908a-790f2080b0a2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:42.395488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a195b54-5cb6-4f63-b3e6-bd6bc9552468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:42.396613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a195b54-5cb6-4f63-b3e6-bd6bc9552468 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:42.397117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a195b54-5cb6-4f63-b3e6-bd6bc9552468 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:42.397699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a195b54-5cb6-4f63-b3e6-bd6bc9552468 None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:42.399965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a195b54-5cb6-4f63-b3e6-bd6bc9552468 None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:42.402856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a195b54-5cb6-4f63-b3e6-bd6bc9552468 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:42.406330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 71/148] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:42 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:42.414741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9d4f3-4f05-4eb7-b583-c77d97978e69 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:42.434855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:42.654486 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9d4f3-4f05-4eb7-b583-c77d97978e69 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:42.705018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b6d96-94f0-413c-9a66-cefe59185f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:42.705713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b6d96-94f0-413c-9a66-cefe59185f97 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:42.706008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b6d96-94f0-413c-9a66-cefe59185f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:42.706294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b6d96-94f0-413c-9a66-cefe59185f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:42.836687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b6d96-94f0-413c-9a66-cefe59185f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:42.839593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b6d96-94f0-413c-9a66-cefe59185f97 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:42.861983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:43.004485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a195b54-5cb6-4f63-b3e6-bd6bc9552468 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:43.058430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 78/149] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:43.065173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b6d96-94f0-413c-9a66-cefe59185f97 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:43.301276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a195b54-5cb6-4f63-b3e6-bd6bc9552468 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:43.329519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:43.361079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b6d96-94f0-413c-9a66-cefe59185f97 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:43.379939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:43.399437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e495e-6b8b-48a3-a237-11fed98a266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:43.400532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e495e-6b8b-48a3-a237-11fed98a266f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:43.401040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e495e-6b8b-48a3-a237-11fed98a266f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:43.401566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e495e-6b8b-48a3-a237-11fed98a266f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:43.435743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e495e-6b8b-48a3-a237-11fed98a266f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-540e495e-6b8b-48a3-a237-11fed98a266f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:43.450987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:43.451929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:15:43.452799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:15:43.453383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 72/150] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:43.461728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e495e-6b8b-48a3-a237-11fed98a266f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:43.477151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b73cf0-cb5d-43d9-ac0b-651be9d380f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:43.478294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b73cf0-cb5d-43d9-ac0b-651be9d380f3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:43.478812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b73cf0-cb5d-43d9-ac0b-651be9d380f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:43.479312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b73cf0-cb5d-43d9-ac0b-651be9d380f3 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:43.486954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b73cf0-cb5d-43d9-ac0b-651be9d380f3 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:43.490420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b73cf0-cb5d-43d9-ac0b-651be9d380f3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:43.493801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 79/151] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:43 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:43.500625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b73cf0-cb5d-43d9-ac0b-651be9d380f3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:43.665180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e495e-6b8b-48a3-a237-11fed98a266f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:43.765767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:43.766473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:43.766757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:43.767033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:43.767717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:43.768021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:43.770017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:43.793274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 73/152] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:43.798895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:43.854202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b73cf0-cb5d-43d9-ac0b-651be9d380f3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:43.885853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:43.960652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:43.961299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:43.961610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:43.961888 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:43.962691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] RBAC: Adding query filter params (role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=dda4727fec9747b4b20cb990f7305e3d, user.id=02a67abf7e2341ab9df44025a4aa90ee) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:43.963003 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:43.964547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:44.000987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 80/153] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:43 2022] GET /identity/v3/role_assignments?user.id=02a67abf7e2341ab9df44025a4aa90ee&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 608 bytes in 501 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:44.078718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d76ca7-5c92-4053-8ce7-b9245d7818d6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:44.123563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cadbc-dbdd-4dff-97a2-1e8630eb2829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:44.124229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cadbc-dbdd-4dff-97a2-1e8630eb2829 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:44.124513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cadbc-dbdd-4dff-97a2-1e8630eb2829 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:44.124790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cadbc-dbdd-4dff-97a2-1e8630eb2829 None admin] PATH_INFO: `/v3/users/32f3633da25b4bea87035e4ebb60a78d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:44.128009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cadbc-dbdd-4dff-97a2-1e8630eb2829 None admin] RBAC: Authorizing `identity:get_user(user_id=32f3633da25b4bea87035e4ebb60a78d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:44.130471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cadbc-dbdd-4dff-97a2-1e8630eb2829 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:44.132779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 74/154] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:43 2022] GET /identity/v3/users/32f3633da25b4bea87035e4ebb60a78d => generated 289 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:44.138437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:44.198869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:44.280183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5869ad5-fafa-4da9-ac02-9e74f85f4c16 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:44.329345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45944ab-0b39-4761-afff-2ced10e76dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:44.330043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45944ab-0b39-4761-afff-2ced10e76dcd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:44.330326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45944ab-0b39-4761-afff-2ced10e76dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:44.330604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45944ab-0b39-4761-afff-2ced10e76dcd None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:44.332477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45944ab-0b39-4761-afff-2ced10e76dcd None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:44.334153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45944ab-0b39-4761-afff-2ced10e76dcd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:44.336029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 81/155] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:44 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:44.342495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cadbc-dbdd-4dff-97a2-1e8630eb2829 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:44.512063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cadbc-dbdd-4dff-97a2-1e8630eb2829 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:44.513896 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:44.556494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee968-2f46-491b-aa20-23f930f86216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:44.557114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee968-2f46-491b-aa20-23f930f86216 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:44.557395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee968-2f46-491b-aa20-23f930f86216 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:44.557701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee968-2f46-491b-aa20-23f930f86216 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d/users/32f3633da25b4bea87035e4ebb60a78d/roles/cd3aa49828434199911a5124c59be2a4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:44.563075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee968-2f46-491b-aa20-23f930f86216 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda4727fec9747b4b20cb990f7305e3d, user_id=32f3633da25b4bea87035e4ebb60a78d, role_id=cd3aa49828434199911a5124c59be2a4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:44.564605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee968-2f46-491b-aa20-23f930f86216 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:44.587654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 75/156] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:15:44 2022] PUT /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d/users/32f3633da25b4bea87035e4ebb60a78d/roles/cd3aa49828434199911a5124c59be2a4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:15:45.517876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14320283-f819-4509-b16b-2fba92da8bf0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:45.518413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14320283-f819-4509-b16b-2fba92da8bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:45.518841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14320283-f819-4509-b16b-2fba92da8bf0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:45.522031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 82/157] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:45.528837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f4b08-c225-4049-a02c-66da695d0853 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:45.529387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f4b08-c225-4049-a02c-66da695d0853 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:45.529831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f4b08-c225-4049-a02c-66da695d0853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:45.573880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c84f4b08-c225-4049-a02c-66da695d0853 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:45.576015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c84f4b08-c225-4049-a02c-66da695d0853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:45.739961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 76/158] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:46.109553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b437-5394-455a-b7ad-62cf22f3d791 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.109968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b437-5394-455a-b7ad-62cf22f3d791 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.110420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9b437-5394-455a-b7ad-62cf22f3d791 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:46.113559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 83/159] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:46.122253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b4162-acd1-4bcc-98eb-411137c50040 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.122722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b4162-acd1-4bcc-98eb-411137c50040 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.123187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b4162-acd1-4bcc-98eb-411137c50040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:46.134485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e797-820d-4c5b-9bd5-be6cfdbb5f80 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.134980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e797-820d-4c5b-9bd5-be6cfdbb5f80 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.135535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e797-820d-4c5b-9bd5-be6cfdbb5f80 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:46.138003 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 84/160] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:46.169995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4e0b4162-acd1-4bcc-98eb-411137c50040 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:46.171921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4e0b4162-acd1-4bcc-98eb-411137c50040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:46.181837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a85d-f9fd-4257-be5a-beb7eb50a267 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.182336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a85d-f9fd-4257-be5a-beb7eb50a267 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.182839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150a85d-f9fd-4257-be5a-beb7eb50a267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:46.209375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2150a85d-f9fd-4257-be5a-beb7eb50a267 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:46.215560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2150a85d-f9fd-4257-be5a-beb7eb50a267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:46.374334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 85/161] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:46.439172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 77/162] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:46.508345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150a85d-f9fd-4257-be5a-beb7eb50a267 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:46.559049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:46.621091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd760d0d-2163-4144-aa0b-6296cf7bc5b4 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.621652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd760d0d-2163-4144-aa0b-6296cf7bc5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.622112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd760d0d-2163-4144-aa0b-6296cf7bc5b4 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:46.624885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 78/163] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:46.643129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb547e-b316-4277-b0d5-a54511ec8b2c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.643634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb547e-b316-4277-b0d5-a54511ec8b2c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.644667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb547e-b316-4277-b0d5-a54511ec8b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:46.665876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150a85d-f9fd-4257-be5a-beb7eb50a267 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:46.713683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9feb547e-b316-4277-b0d5-a54511ec8b2c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:46.716070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9feb547e-b316-4277-b0d5-a54511ec8b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:46.777613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdd44d-e494-46b5-9ff5-385a8a46e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:46.777815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:46.778740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd44d-e494-46b5-9ff5-385a8a46e48c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.779291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd44d-e494-46b5-9ff5-385a8a46e48c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.779784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd44d-e494-46b5-9ff5-385a8a46e48c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:46.812184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdd44d-e494-46b5-9ff5-385a8a46e48c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d6bdd44d-e494-46b5-9ff5-385a8a46e48c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:46.818264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:46.819264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:46.820065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:46.822793 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 86/164] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:15:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:46.836783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdd44d-e494-46b5-9ff5-385a8a46e48c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:46.961961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 79/165] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:46.964244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00358269-0fdf-4748-b18a-2201f34a4738 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.964731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00358269-0fdf-4748-b18a-2201f34a4738 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.965223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00358269-0fdf-4748-b18a-2201f34a4738 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:46.967832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 80/166] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:46.989088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcde74-bd78-4ae7-bc04-495b71cb92db None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:46.989632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcde74-bd78-4ae7-bc04-495b71cb92db None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:46.990109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcde74-bd78-4ae7-bc04-495b71cb92db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:47.038015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e1dcde74-bd78-4ae7-bc04-495b71cb92db None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:47.040413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e1dcde74-bd78-4ae7-bc04-495b71cb92db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:47.082921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:47.090309 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdd44d-e494-46b5-9ff5-385a8a46e48c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:47.150953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:47.151970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:47.152457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:47.152936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:47.153872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:47.154506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:47.156182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:47.181341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 87/167] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:47.182574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:47.243244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 81/168] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:47.244081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcde74-bd78-4ae7-bc04-495b71cb92db None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:47.415635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ac300-0abb-4b81-a954-6e1a23f55b7e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:47.506504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:47.542246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac7985-9af0-42ef-9ffa-4b69aed86152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:47.542923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac7985-9af0-42ef-9ffa-4b69aed86152 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:47.543211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac7985-9af0-42ef-9ffa-4b69aed86152 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:47.543493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac7985-9af0-42ef-9ffa-4b69aed86152 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:47.545072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcde74-bd78-4ae7-bc04-495b71cb92db None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:47.578192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac7985-9af0-42ef-9ffa-4b69aed86152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cbac7985-9af0-42ef-9ffa-4b69aed86152 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:47.584174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:47.586339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 88/169] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:47.587947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac7985-9af0-42ef-9ffa-4b69aed86152 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:47.619933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9317e3b-8855-4710-9a2c-90e9c22baf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:47.621422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9317e3b-8855-4710-9a2c-90e9c22baf66 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:47.623139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9317e3b-8855-4710-9a2c-90e9c22baf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:47.623864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9317e3b-8855-4710-9a2c-90e9c22baf66 None admin] PATH_INFO: `/v3/users/72593536090c47589cfad801ff3da440` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:47.626335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9317e3b-8855-4710-9a2c-90e9c22baf66 None admin] RBAC: Authorizing `identity:get_user(user_id=72593536090c47589cfad801ff3da440)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:47.630126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9317e3b-8855-4710-9a2c-90e9c22baf66 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:47.633993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 82/170] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:47 2022] GET /identity/v3/users/72593536090c47589cfad801ff3da440 => generated 247 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:47.635702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9317e3b-8855-4710-9a2c-90e9c22baf66 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:47.650905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:47.830935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9317e3b-8855-4710-9a2c-90e9c22baf66 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:47.832347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac7985-9af0-42ef-9ffa-4b69aed86152 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:47.897639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0755c82-e37b-43bd-8822-6f68ee7773d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:47.897901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d1209-1b3b-4071-b29f-1ef39aceb0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:47.898786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d1209-1b3b-4071-b29f-1ef39aceb0fb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:47.899271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0755c82-e37b-43bd-8822-6f68ee7773d1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:47.899474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d1209-1b3b-4071-b29f-1ef39aceb0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:47.900044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0755c82-e37b-43bd-8822-6f68ee7773d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:47.900194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d1209-1b3b-4071-b29f-1ef39aceb0fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:47.900863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0755c82-e37b-43bd-8822-6f68ee7773d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:47.920261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0755c82-e37b-43bd-8822-6f68ee7773d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:47.922099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d1209-1b3b-4071-b29f-1ef39aceb0fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d0755c82-e37b-43bd-8822-6f68ee7773d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:47.930370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:47.932158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:47.933714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-205d1209-1b3b-4071-b29f-1ef39aceb0fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:47.934901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:47.943141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 89/171] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:15:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:47.945492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d1209-1b3b-4071-b29f-1ef39aceb0fb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:47.951484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 83/172] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:15:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:47.952168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0755c82-e37b-43bd-8822-6f68ee7773d1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:47.979252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:48.091723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:48.125139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0755c82-e37b-43bd-8822-6f68ee7773d1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.128380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d1209-1b3b-4071-b29f-1ef39aceb0fb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.174255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a81a54-cd87-4b0a-9869-1637acdf4da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.174869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a81a54-cd87-4b0a-9869-1637acdf4da7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.175826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a81a54-cd87-4b0a-9869-1637acdf4da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.176228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a81a54-cd87-4b0a-9869-1637acdf4da7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.183703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.184533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.184940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.185372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.186774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:48.187217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:48.189612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:48.200815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 90/173] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:48.206441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:48.270467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a81a54-cd87-4b0a-9869-1637acdf4da7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:48.272148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a81a54-cd87-4b0a-9869-1637acdf4da7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:48.288922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:48.329804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354229bc-1092-47cb-a306-17656df32d20 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.379604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d86926-5f3b-408f-8d73-4bb53f8c22b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.380169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d86926-5f3b-408f-8d73-4bb53f8c22b0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.380457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d86926-5f3b-408f-8d73-4bb53f8c22b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.380736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d86926-5f3b-408f-8d73-4bb53f8c22b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.390613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:48.395718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d86926-5f3b-408f-8d73-4bb53f8c22b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b6d86926-5f3b-408f-8d73-4bb53f8c22b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:48.402518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:48.403571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:48.404416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 91/174] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:15:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:48.405183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d86926-5f3b-408f-8d73-4bb53f8c22b0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:48.437988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 84/175] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:48.437988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a81a54-cd87-4b0a-9869-1637acdf4da7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:48.542468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d86926-5f3b-408f-8d73-4bb53f8c22b0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.558041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a81a54-cd87-4b0a-9869-1637acdf4da7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.570532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:48.589295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef669e9e-052e-4bdf-9bcb-faff0f3a321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.590215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef669e9e-052e-4bdf-9bcb-faff0f3a321a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.590403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef669e9e-052e-4bdf-9bcb-faff0f3a321a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.590902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef669e9e-052e-4bdf-9bcb-faff0f3a321a None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.592135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef669e9e-052e-4bdf-9bcb-faff0f3a321a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:48.592700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef669e9e-052e-4bdf-9bcb-faff0f3a321a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:48.595405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef669e9e-052e-4bdf-9bcb-faff0f3a321a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:48.599636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311c14b-7e63-4e74-a01c-a9f6af65789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.600264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311c14b-7e63-4e74-a01c-a9f6af65789a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.600541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311c14b-7e63-4e74-a01c-a9f6af65789a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.600811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311c14b-7e63-4e74-a01c-a9f6af65789a None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.601503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311c14b-7e63-4e74-a01c-a9f6af65789a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:48.601808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311c14b-7e63-4e74-a01c-a9f6af65789a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:48.603192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9311c14b-7e63-4e74-a01c-a9f6af65789a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:48.608778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 92/176] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:48.610087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b4a3e-b2eb-4317-8a82-e2030f7873e1 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.610370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b4a3e-b2eb-4317-8a82-e2030f7873e1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.610645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b4a3e-b2eb-4317-8a82-e2030f7873e1 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.613499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 93/177] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:48.613499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b4a3e-b2eb-4317-8a82-e2030f7873e1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:48.613731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 85/178] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:15:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:48.614572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19967ae4-1c66-4e00-a426-42131d3d0f21 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.614867 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19967ae4-1c66-4e00-a426-42131d3d0f21 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.615145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19967ae4-1c66-4e00-a426-42131d3d0f21 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.616762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 86/179] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:48.618959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19967ae4-1c66-4e00-a426-42131d3d0f21 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:48.690315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:48.744136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b4a3e-b2eb-4317-8a82-e2030f7873e1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.745673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19967ae4-1c66-4e00-a426-42131d3d0f21 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.785425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.786095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.786506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.786668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.787314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:48.787619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:48.789096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:48.789565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83023fd2-74fa-4597-9ade-e6e1d4e9aafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.790246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023fd2-74fa-4597-9ade-e6e1d4e9aafb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.790523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023fd2-74fa-4597-9ade-e6e1d4e9aafb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.790806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83023fd2-74fa-4597-9ade-e6e1d4e9aafb None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.792084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83023fd2-74fa-4597-9ade-e6e1d4e9aafb None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:48.793647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83023fd2-74fa-4597-9ade-e6e1d4e9aafb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:48.795398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 94/180] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:48 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:48.796254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83023fd2-74fa-4597-9ade-e6e1d4e9aafb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:48.804724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 87/181] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:15:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:48.806365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:48.849889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:48.938378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7501c4-dae1-4753-b1c7-ae4d65c6bfd2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.952500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83023fd2-74fa-4597-9ade-e6e1d4e9aafb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:48.985737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c98efa4-2a45-40a3-a9ad-7b153572994e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.986409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98efa4-2a45-40a3-a9ad-7b153572994e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.986689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98efa4-2a45-40a3-a9ad-7b153572994e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.986806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:48.986974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c98efa4-2a45-40a3-a9ad-7b153572994e None admin] PATH_INFO: `/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.995640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0b959-0b95-41e5-b828-2fff7ce78198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:48.996306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0b959-0b95-41e5-b828-2fff7ce78198 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:48.996593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0b959-0b95-41e5-b828-2fff7ce78198 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:48.996877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0b959-0b95-41e5-b828-2fff7ce78198 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:48.998239 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0b959-0b95-41e5-b828-2fff7ce78198 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:48.999771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0b959-0b95-41e5-b828-2fff7ce78198 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:49.001635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 95/182] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:48 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:49.002928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da3883-1f96-43cd-a9c2-ddee20c8de2d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:49.003813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da3883-1f96-43cd-a9c2-ddee20c8de2d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:49.004206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da3883-1f96-43cd-a9c2-ddee20c8de2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:49.012073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98efa4-2a45-40a3-a9ad-7b153572994e None admin] RBAC: Authorizing `identity:get_user(user_id=58c5b7efbca845bd9d5d1c8cc8f50b80)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:49.013951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c98efa4-2a45-40a3-a9ad-7b153572994e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:49.015543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 88/183] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:48 2022] GET /identity/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:49.016995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9763f5-be9d-416e-98d6-adb7d8bd2f20 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:49.017284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9763f5-be9d-416e-98d6-adb7d8bd2f20 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:49.017602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9763f5-be9d-416e-98d6-adb7d8bd2f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:49.028805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-93da3883-1f96-43cd-a9c2-ddee20c8de2d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:49.030146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-93da3883-1f96-43cd-a9c2-ddee20c8de2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:49.038726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9a9763f5-be9d-416e-98d6-adb7d8bd2f20 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:49.039980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9a9763f5-be9d-416e-98d6-adb7d8bd2f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:49.123312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 96/184] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:49.124394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da3883-1f96-43cd-a9c2-ddee20c8de2d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:49.143083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 89/185] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:49.145011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9763f5-be9d-416e-98d6-adb7d8bd2f20 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:49.200289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:49.319134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9763f5-be9d-416e-98d6-adb7d8bd2f20 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:49.328975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da3883-1f96-43cd-a9c2-ddee20c8de2d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:49.363759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296012-0637-4d53-a281-1eee6e56ba57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:49.364943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296012-0637-4d53-a281-1eee6e56ba57 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:49.365438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296012-0637-4d53-a281-1eee6e56ba57 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:49.365947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296012-0637-4d53-a281-1eee6e56ba57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:49.386878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:49.400690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2521a-263f-43aa-9334-07221cf16cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:49.401979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2521a-263f-43aa-9334-07221cf16cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:49.402479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2521a-263f-43aa-9334-07221cf16cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:49.403088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2521a-263f-43aa-9334-07221cf16cd2 None admin] PATH_INFO: `/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:49.405251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2521a-263f-43aa-9334-07221cf16cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=58c5b7efbca845bd9d5d1c8cc8f50b80)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:49.408148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2521a-263f-43aa-9334-07221cf16cd2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:49.412087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 97/186] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:49 2022] GET /identity/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80 => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:49.413495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2521a-263f-43aa-9334-07221cf16cd2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:49.450244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed296012-0637-4d53-a281-1eee6e56ba57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:49.451745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed296012-0637-4d53-a281-1eee6e56ba57 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:49.479249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:49.557507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2521a-263f-43aa-9334-07221cf16cd2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:49.596942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 90/187] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:49.596942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296012-0637-4d53-a281-1eee6e56ba57 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:49.626029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86172840-57be-4e71-bb7f-f159e39206eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:49.627746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172840-57be-4e71-bb7f-f159e39206eb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:49.628419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172840-57be-4e71-bb7f-f159e39206eb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:49.628905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86172840-57be-4e71-bb7f-f159e39206eb None admin] PATH_INFO: `/v3/users/02a67abf7e2341ab9df44025a4aa90ee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:49.632289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86172840-57be-4e71-bb7f-f159e39206eb None admin] RBAC: Authorizing `identity:get_user(user_id=02a67abf7e2341ab9df44025a4aa90ee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:49.636474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86172840-57be-4e71-bb7f-f159e39206eb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:49.644816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 98/188] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:49 2022] GET /identity/v3/users/02a67abf7e2341ab9df44025a4aa90ee => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:49.644816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86172840-57be-4e71-bb7f-f159e39206eb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:49.730542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:49.744603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296012-0637-4d53-a281-1eee6e56ba57 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:49.773930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:49.774916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86172840-57be-4e71-bb7f-f159e39206eb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:49.789197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:49.790026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:49.790454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:49.790922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:49.791853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=dda4727fec9747b4b20cb990f7305e3d, user.id=72593536090c47589cfad801ff3da440) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:49.792247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:49.794031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:49.801684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 91/189] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:49 2022] GET /identity/v3/role_assignments?user.id=72593536090c47589cfad801ff3da440&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:49.802695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:49.826991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afff82e-abde-4314-be90-c64ae77d78e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:49.827812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff82e-abde-4314-be90-c64ae77d78e7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:49.828261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff82e-abde-4314-be90-c64ae77d78e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:49.828669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff82e-abde-4314-be90-c64ae77d78e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:49.923086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afff82e-abde-4314-be90-c64ae77d78e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:49.924557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afff82e-abde-4314-be90-c64ae77d78e7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:49.943753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a77d-0246-4ad9-8fab-ed2aa253ba0c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:49.993611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb708b0-fccb-4e93-9086-fac2fb3b5ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:49.994258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb708b0-fccb-4e93-9086-fac2fb3b5ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:49.994590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb708b0-fccb-4e93-9086-fac2fb3b5ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:49.994872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb708b0-fccb-4e93-9086-fac2fb3b5ef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.015852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:50.078385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 99/190] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:50.079726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afff82e-abde-4314-be90-c64ae77d78e7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.083064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:50.110935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb708b0-fccb-4e93-9086-fac2fb3b5ef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:50.114440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb708b0-fccb-4e93-9086-fac2fb3b5ef1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:50.236431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afff82e-abde-4314-be90-c64ae77d78e7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:50.276370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 92/191] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:50.276888 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33425e4-0848-499f-be95-ebd0eeeb3bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:50.277649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33425e4-0848-499f-be95-ebd0eeeb3bfb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:50.277942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33425e4-0848-499f-be95-ebd0eeeb3bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:50.278223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33425e4-0848-499f-be95-ebd0eeeb3bfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.278794 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb708b0-fccb-4e93-9086-fac2fb3b5ef1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.279629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33425e4-0848-499f-be95-ebd0eeeb3bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:50.281853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33425e4-0848-499f-be95-ebd0eeeb3bfb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:50.283548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 100/192] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:50.285618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33425e4-0848-499f-be95-ebd0eeeb3bfb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.318742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:50.396511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:50.397661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33425e4-0848-499f-be95-ebd0eeeb3bfb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:50.415996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb708b0-fccb-4e93-9086-fac2fb3b5ef1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:50.436565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dfb60-0b32-45c2-bb99-9cd0f9a42109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:50.437189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dfb60-0b32-45c2-bb99-9cd0f9a42109 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:50.437499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dfb60-0b32-45c2-bb99-9cd0f9a42109 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:50.437809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dfb60-0b32-45c2-bb99-9cd0f9a42109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.453467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593f68c-6d57-4064-80fc-f6450a8247eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:50.454137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593f68c-6d57-4064-80fc-f6450a8247eb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:50.454415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593f68c-6d57-4064-80fc-f6450a8247eb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:50.454690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593f68c-6d57-4064-80fc-f6450a8247eb None admin] PATH_INFO: `/v3/projects/9c725885325642f88ad12b0f1492310d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.455846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593f68c-6d57-4064-80fc-f6450a8247eb None admin] RBAC: Authorizing `identity:get_project(project_id=9c725885325642f88ad12b0f1492310d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:50.458082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593f68c-6d57-4064-80fc-f6450a8247eb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:50.459832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 93/193] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/projects/9c725885325642f88ad12b0f1492310d => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:50.460731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593f68c-6d57-4064-80fc-f6450a8247eb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.520189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dfb60-0b32-45c2-bb99-9cd0f9a42109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:50.522204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dfb60-0b32-45c2-bb99-9cd0f9a42109 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:50.558488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:50.579047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593f68c-6d57-4064-80fc-f6450a8247eb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:50.617717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a15ed4-2ac1-4fc1-b838-10ff90751607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:50.618372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a15ed4-2ac1-4fc1-b838-10ff90751607 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:50.618658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a15ed4-2ac1-4fc1-b838-10ff90751607 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:50.618935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a15ed4-2ac1-4fc1-b838-10ff90751607 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.634950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a15ed4-2ac1-4fc1-b838-10ff90751607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:50.647441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 101/194] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:15:50.647441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dfb60-0b32-45c2-bb99-9cd0f9a42109 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-73a15ed4-2ac1-4fc1-b838-10ff90751607 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:50.648053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:50.649115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 94/195] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:50.650155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a15ed4-2ac1-4fc1-b838-10ff90751607 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.673038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:50.769762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a15ed4-2ac1-4fc1-b838-10ff90751607 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:50.770039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dfb60-0b32-45c2-bb99-9cd0f9a42109 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:50.800796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:50.805205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094edce-2051-4283-a513-0a51635b145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:50.805889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094edce-2051-4283-a513-0a51635b145d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:50.806164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094edce-2051-4283-a513-0a51635b145d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:50.806442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094edce-2051-4283-a513-0a51635b145d None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.807513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cfbe0-d65f-45b4-adb3-589fe291785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:50.807693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094edce-2051-4283-a513-0a51635b145d None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:50.808165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cfbe0-d65f-45b4-adb3-589fe291785e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:50.808453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cfbe0-d65f-45b4-adb3-589fe291785e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:50.808736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cfbe0-d65f-45b4-adb3-589fe291785e None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.809092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094edce-2051-4283-a513-0a51635b145d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:50.810020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cfbe0-d65f-45b4-adb3-589fe291785e None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:50.810802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 95/196] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:50.811500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cfbe0-d65f-45b4-adb3-589fe291785e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:50.811645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094edce-2051-4283-a513-0a51635b145d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.819037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 102/197] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:50.819037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cfbe0-d65f-45b4-adb3-589fe291785e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.910663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:50.930672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094edce-2051-4283-a513-0a51635b145d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:50.937080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cfbe0-d65f-45b4-adb3-589fe291785e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:50.967263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d84a8a-6322-4627-941b-9239464f6866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:50.967897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d84a8a-6322-4627-941b-9239464f6866 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:50.968177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d84a8a-6322-4627-941b-9239464f6866 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:50.968455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d84a8a-6322-4627-941b-9239464f6866 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.969120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d84a8a-6322-4627-941b-9239464f6866 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:50.970726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d84a8a-6322-4627-941b-9239464f6866 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:50.972472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:50.973159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:50.973551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:50.973903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:50.974755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] RBAC: Adding query filter params (role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=9c725885325642f88ad12b0f1492310d, user.id=02a67abf7e2341ab9df44025a4aa90ee) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:50.975134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:50.976661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:50.986203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 103/198] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/role_assignments?user.id=02a67abf7e2341ab9df44025a4aa90ee&scope.project.id=9c725885325642f88ad12b0f1492310d&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:50.997685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:50.997788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 96/199] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:15:50 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:15:51.009483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d84a8a-6322-4627-941b-9239464f6866 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:51.130975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:51.143546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d84a8a-6322-4627-941b-9239464f6866 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:51.191758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51429148-9190-49db-8ee4-fce8c4e34130 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:51.195248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:51.195729 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:51.196116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:51.196560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:51.197266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:51.197631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:51.199096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:51.208774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:51.213929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 97/200] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:51.215174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:51.242506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757cb44-e90a-4f26-8cf0-c7e52477c7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:51.243281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757cb44-e90a-4f26-8cf0-c7e52477c7bd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:51.243574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757cb44-e90a-4f26-8cf0-c7e52477c7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:51.243857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757cb44-e90a-4f26-8cf0-c7e52477c7bd None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:51.245872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757cb44-e90a-4f26-8cf0-c7e52477c7bd None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:51.247725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757cb44-e90a-4f26-8cf0-c7e52477c7bd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:51.250460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 104/201] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:50 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:51.251431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757cb44-e90a-4f26-8cf0-c7e52477c7bd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:51.361728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4399810e-a98c-4f49-8061-bfbc02115d5b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:51.406369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757cb44-e90a-4f26-8cf0-c7e52477c7bd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:51.407861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:51.408914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:51.409468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:51.411002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:51.412353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] RBAC: Adding query filter params (role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=5a96496b71e840f8b6e17be47512394b, user.id=58c5b7efbca845bd9d5d1c8cc8f50b80) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:51.412866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:51.415760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:51.430651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 98/202] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:51 2022] GET /identity/v3/role_assignments?user.id=58c5b7efbca845bd9d5d1c8cc8f50b80&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:51.432106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:51.454816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:51.465113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:51.465811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:51.466101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:51.466394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:51.467169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=5a96496b71e840f8b6e17be47512394b, user.id=58c5b7efbca845bd9d5d1c8cc8f50b80) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:51.467493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:51.469023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:51.475850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 105/203] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:51 2022] GET /identity/v3/role_assignments?user.id=58c5b7efbca845bd9d5d1c8cc8f50b80&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:51.476548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:51.555984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:51.619261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7396ea2-89ca-4671-b66b-1796e4aaa923 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:51.623632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb80fa2-28ae-449b-b7df-4b76086af999 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:51.668052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba7b9-4066-4974-99db-d06b83c4ec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:51.669773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba7b9-4066-4974-99db-d06b83c4ec9c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:51.670279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba7b9-4066-4974-99db-d06b83c4ec9c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:51.670761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba7b9-4066-4974-99db-d06b83c4ec9c None admin] PATH_INFO: `/v3/users/32f3633da25b4bea87035e4ebb60a78d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:51.672731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ba7b9-4066-4974-99db-d06b83c4ec9c None admin] RBAC: Authorizing `identity:get_user(user_id=32f3633da25b4bea87035e4ebb60a78d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:51.674870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ba7b9-4066-4974-99db-d06b83c4ec9c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:51.679205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 99/204] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:51 2022] GET /identity/v3/users/32f3633da25b4bea87035e4ebb60a78d => generated 289 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:51.701855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba7b9-4066-4974-99db-d06b83c4ec9c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:51.702503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:51.703461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:51.704034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:51.704479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:51.705725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] RBAC: Adding query filter params (role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=dda4727fec9747b4b20cb990f7305e3d, user.id=02a67abf7e2341ab9df44025a4aa90ee) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:51.706920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:51.710010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:51.723149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 106/205] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:51 2022] GET /identity/v3/role_assignments?user.id=02a67abf7e2341ab9df44025a4aa90ee&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:51.940034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba7b9-4066-4974-99db-d06b83c4ec9c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:52.000953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:52.013130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcafcaa-da22-4e14-8f67-18a5d46357b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:52.014250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcafcaa-da22-4e14-8f67-18a5d46357b9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:52.014742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcafcaa-da22-4e14-8f67-18a5d46357b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:52.015216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcafcaa-da22-4e14-8f67-18a5d46357b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:52.149486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcafcaa-da22-4e14-8f67-18a5d46357b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:52.152255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcafcaa-da22-4e14-8f67-18a5d46357b9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:52.326558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 100/206] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 13:15:52.332924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:52.421087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:52.476786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43213cc9-bbce-4616-bbce-e202ce485451 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:52.519193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0f349-cc6e-4c2d-9634-87099fad299c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:52.520276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0f349-cc6e-4c2d-9634-87099fad299c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:52.520765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0f349-cc6e-4c2d-9634-87099fad299c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:52.521244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0f349-cc6e-4c2d-9634-87099fad299c None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:52.522679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c0f349-cc6e-4c2d-9634-87099fad299c None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:52.524712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c0f349-cc6e-4c2d-9634-87099fad299c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:52.526742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 107/207] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:52 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:52.535928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcafcaa-da22-4e14-8f67-18a5d46357b9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:52.591792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:52.679426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcafcaa-da22-4e14-8f67-18a5d46357b9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:52.719973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc6bd2-30e9-4766-9eb3-6087111eada3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:52.720986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc6bd2-30e9-4766-9eb3-6087111eada3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:52.721509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc6bd2-30e9-4766-9eb3-6087111eada3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:52.722017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc6bd2-30e9-4766-9eb3-6087111eada3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:52.723743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc6bd2-30e9-4766-9eb3-6087111eada3 None admin] RBAC: Adding query filter params (role.id=cd3aa49828434199911a5124c59be2a4, scope.project.id=dda4727fec9747b4b20cb990f7305e3d, user.id=32f3633da25b4bea87035e4ebb60a78d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:52.724269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc6bd2-30e9-4766-9eb3-6087111eada3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:52.726388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc6bd2-30e9-4766-9eb3-6087111eada3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:52.733571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 101/208] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:52 2022] GET /identity/v3/role_assignments?user.id=32f3633da25b4bea87035e4ebb60a78d&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=cd3aa49828434199911a5124c59be2a4 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:54.489010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a247a-7a13-4100-b422-2acaef61bda8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:54.489010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a247a-7a13-4100-b422-2acaef61bda8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:54.489010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a247a-7a13-4100-b422-2acaef61bda8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:54.490416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 108/209] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:54.496650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a30e3-d58f-4bf6-b7af-4480b28ee4d6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:54.497677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a30e3-d58f-4bf6-b7af-4480b28ee4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:54.498184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a30e3-d58f-4bf6-b7af-4480b28ee4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:54.520002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f09a30e3-d58f-4bf6-b7af-4480b28ee4d6 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:54.521667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f09a30e3-d58f-4bf6-b7af-4480b28ee4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:54.602508 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 102/210] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:54.855227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029b169-c766-4d18-9e33-bb92753af076 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:54.856258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029b169-c766-4d18-9e33-bb92753af076 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:54.857053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029b169-c766-4d18-9e33-bb92753af076 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:54.860542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 109/211] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:54.870095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e711e-a74a-42fa-a309-fd85cbfb61ec None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:54.870737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e711e-a74a-42fa-a309-fd85cbfb61ec None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:54.871337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e711e-a74a-42fa-a309-fd85cbfb61ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:54.898803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ed8e711e-a74a-42fa-a309-fd85cbfb61ec None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:54.900491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ed8e711e-a74a-42fa-a309-fd85cbfb61ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:54.902917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029b169-c766-4d18-9e33-bb92753af076 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:54.935735 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:55.034012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 103/212] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:55.071429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029b169-c766-4d18-9e33-bb92753af076 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:55.105580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:55.116628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea9d44-3e11-4310-b02b-e00d7a60cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:55.117424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea9d44-3e11-4310-b02b-e00d7a60cae7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.117746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea9d44-3e11-4310-b02b-e00d7a60cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.118024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea9d44-3e11-4310-b02b-e00d7a60cae7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.124254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7156c-b9c4-491f-9351-bcdfaa0edcbc None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.125142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7156c-b9c4-491f-9351-bcdfaa0edcbc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.125914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7156c-b9c4-491f-9351-bcdfaa0edcbc None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.133376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea9d44-3e11-4310-b02b-e00d7a60cae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:55.136934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 104/213] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:55 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-dbea9d44-3e11-4310-b02b-e00d7a60cae7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:55.139707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:55.141340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:15:55.142908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:55.145948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80a325-aad3-454b-8cbd-705d300a57ec None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.146196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 110/214] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:15:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:15:55.146419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80a325-aad3-454b-8cbd-705d300a57ec None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.146419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80a325-aad3-454b-8cbd-705d300a57ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.150181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea9d44-3e11-4310-b02b-e00d7a60cae7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:55.174152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6c80a325-aad3-454b-8cbd-705d300a57ec None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:55.177354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6c80a325-aad3-454b-8cbd-705d300a57ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:55.297191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 105/215] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:55.297191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38aa843-96cd-4572-9b28-285e395bbcd7 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.297191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38aa843-96cd-4572-9b28-285e395bbcd7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.297191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38aa843-96cd-4572-9b28-285e395bbcd7 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.297191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 106/216] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:55.304519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0149-3681-4355-84d8-6911e8f4090d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.305000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0149-3681-4355-84d8-6911e8f4090d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.305491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb0149-3681-4355-84d8-6911e8f4090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.334646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea9d44-3e11-4310-b02b-e00d7a60cae7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:55.337281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f9fb0149-3681-4355-84d8-6911e8f4090d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:55.346418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f9fb0149-3681-4355-84d8-6911e8f4090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:55.419181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:55.419743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.420185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.420302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.421218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:55.421572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:55.423115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:55.429743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 107/217] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:15:55.430970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb0149-3681-4355-84d8-6911e8f4090d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:55.445103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 111/218] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:55.454519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:55.458516 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:55.530867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:55.597482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48ab44-4261-4f38-9fc9-f519fdbc5b7c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:55.598786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb0149-3681-4355-84d8-6911e8f4090d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:55.643695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1ef97-227a-43ab-b38a-5abf9942cd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:55.644357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1ef97-227a-43ab-b38a-5abf9942cd6e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.645224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1ef97-227a-43ab-b38a-5abf9942cd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.645549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1ef97-227a-43ab-b38a-5abf9942cd6e None admin] PATH_INFO: `/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.647709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1ef97-227a-43ab-b38a-5abf9942cd6e None admin] RBAC: Authorizing `identity:get_user(user_id=58c5b7efbca845bd9d5d1c8cc8f50b80)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:55.649400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1ef97-227a-43ab-b38a-5abf9942cd6e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:55.651564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 112/219] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:55 2022] GET /identity/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:55.652869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1ef97-227a-43ab-b38a-5abf9942cd6e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:55.670846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d52c3-7578-40a7-ae6e-a0eb93f6ac04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:55.671433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d52c3-7578-40a7-ae6e-a0eb93f6ac04 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.671713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d52c3-7578-40a7-ae6e-a0eb93f6ac04 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.671983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d52c3-7578-40a7-ae6e-a0eb93f6ac04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.673137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d52c3-7578-40a7-ae6e-a0eb93f6ac04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:55.675982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d52c3-7578-40a7-ae6e-a0eb93f6ac04 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:55.678006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 108/220] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:15:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:55.678562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d52c3-7578-40a7-ae6e-a0eb93f6ac04 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:55.786804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:55.813321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d52c3-7578-40a7-ae6e-a0eb93f6ac04 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:55.823217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1ef97-227a-43ab-b38a-5abf9942cd6e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:55.852033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:55.864403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe42691-3f33-42d2-aa14-635c4bed1f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:55.865435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe42691-3f33-42d2-aa14-635c4bed1f8e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.867019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe42691-3f33-42d2-aa14-635c4bed1f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.867019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe42691-3f33-42d2-aa14-635c4bed1f8e None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.868910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe42691-3f33-42d2-aa14-635c4bed1f8e None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:55.870873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe42691-3f33-42d2-aa14-635c4bed1f8e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:55.873851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8499dd9-a775-4991-885d-552f1ac9d5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:55.874151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 109/221] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:55 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:55.874929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499dd9-a775-4991-885d-552f1ac9d5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:55.875278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499dd9-a775-4991-885d-552f1ac9d5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:55.875535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe42691-3f33-42d2-aa14-635c4bed1f8e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:55.875647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499dd9-a775-4991-885d-552f1ac9d5a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:55.899042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8499dd9-a775-4991-885d-552f1ac9d5a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a8499dd9-a775-4991-885d-552f1ac9d5a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:55.906379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:55.910992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 113/222] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:15:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:55.911792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8499dd9-a775-4991-885d-552f1ac9d5a1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:56.034614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe42691-3f33-42d2-aa14-635c4bed1f8e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:56.053149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8499dd9-a775-4991-885d-552f1ac9d5a1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:56.078277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b0ae1-3a5e-423b-9ea5-b56083e59cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:56.079318 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b0ae1-3a5e-423b-9ea5-b56083e59cba None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:56.079803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b0ae1-3a5e-423b-9ea5-b56083e59cba None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:56.080214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b0ae1-3a5e-423b-9ea5-b56083e59cba None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:56.080944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b0ae1-3a5e-423b-9ea5-b56083e59cba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:56.082923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b0ae1-3a5e-423b-9ea5-b56083e59cba None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:56.107087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9081d-cefc-42c5-89a2-011a73628259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:56.108040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9081d-cefc-42c5-89a2-011a73628259 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:56.108425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9081d-cefc-42c5-89a2-011a73628259 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:56.108904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba9081d-cefc-42c5-89a2-011a73628259 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:56.110050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 110/223] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:15:55 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:15:56.110888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b0ae1-3a5e-423b-9ea5-b56083e59cba None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:56.135684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba9081d-cefc-42c5-89a2-011a73628259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:15:56.138328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-eba9081d-cefc-42c5-89a2-011a73628259 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:15:56.141621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:15:56.142860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 114/224] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:15:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:15:56.146488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9081d-cefc-42c5-89a2-011a73628259 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:56.216734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:56.324057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b0ae1-3a5e-423b-9ea5-b56083e59cba None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:56.353550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba9081d-cefc-42c5-89a2-011a73628259 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:56.391416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43bbc-ed4d-48fe-93af-76ee242d62a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:56.393220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43bbc-ed4d-48fe-93af-76ee242d62a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:56.393220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43bbc-ed4d-48fe-93af-76ee242d62a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:56.394089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43bbc-ed4d-48fe-93af-76ee242d62a4 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b/users/58c5b7efbca845bd9d5d1c8cc8f50b80/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:56.396549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc43bbc-ed4d-48fe-93af-76ee242d62a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a96496b71e840f8b6e17be47512394b, user_id=58c5b7efbca845bd9d5d1c8cc8f50b80, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:56.401491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc43bbc-ed4d-48fe-93af-76ee242d62a4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:56.418444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:56.419074 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:56.419417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:56.419713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:56.420517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:56.420829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:56.422315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:56.430417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 111/225] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:15:56 2022] PUT /identity/v3/projects/5a96496b71e840f8b6e17be47512394b/users/58c5b7efbca845bd9d5d1c8cc8f50b80/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:15:56.437807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43bbc-ed4d-48fe-93af-76ee242d62a4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:56.438081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 115/226] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:15:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:56.439999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:56.571913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:56.624694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43bbc-ed4d-48fe-93af-76ee242d62a4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:56.655435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:56.661258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7505548-2ef2-4e28-b19e-fc8201b3ff02 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:56.680272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:56.680996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:56.681295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:56.681615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:56.682943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:56.683307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:56.684892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:56.696944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 112/227] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:15:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:56.703501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:56.721012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68c928-a60b-4d72-afec-3f23183ee0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:56.722609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c928-a60b-4d72-afec-3f23183ee0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:56.723459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c928-a60b-4d72-afec-3f23183ee0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:56.723920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68c928-a60b-4d72-afec-3f23183ee0a5 None admin] PATH_INFO: `/v3/users/1759d68410ef41369c91d7026927f2f2` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:56.745653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68c928-a60b-4d72-afec-3f23183ee0a5 None admin] RBAC: Authorizing `identity:get_user(user_id=1759d68410ef41369c91d7026927f2f2)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:56.748527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68c928-a60b-4d72-afec-3f23183ee0a5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:56.751584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 116/228] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:56 2022] GET /identity/v3/users/1759d68410ef41369c91d7026927f2f2 => generated 297 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:56.758744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68c928-a60b-4d72-afec-3f23183ee0a5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:56.921668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f517-b597-4c46-92ff-29658d1671a8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:56.928302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68c928-a60b-4d72-afec-3f23183ee0a5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:56.979049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:56.993634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3560b0-3e71-4862-aca1-8422337ada6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:56.994710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3560b0-3e71-4862-aca1-8422337ada6d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:56.995306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3560b0-3e71-4862-aca1-8422337ada6d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:56.995776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3560b0-3e71-4862-aca1-8422337ada6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:56.996339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113895a2-cc2e-4387-acfe-8506ad6761eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:56.997020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113895a2-cc2e-4387-acfe-8506ad6761eb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:56.997306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113895a2-cc2e-4387-acfe-8506ad6761eb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:56.997617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113895a2-cc2e-4387-acfe-8506ad6761eb None admin] PATH_INFO: `/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:57.001823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113895a2-cc2e-4387-acfe-8506ad6761eb None admin] RBAC: Authorizing `identity:get_user(user_id=58c5b7efbca845bd9d5d1c8cc8f50b80)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:57.004612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113895a2-cc2e-4387-acfe-8506ad6761eb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:57.007020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 113/229] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:15:56 2022] GET /identity/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80 => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:57.022540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113895a2-cc2e-4387-acfe-8506ad6761eb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:57.076700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:57.226656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3560b0-3e71-4862-aca1-8422337ada6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:57.232592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3560b0-3e71-4862-aca1-8422337ada6d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:57.336442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113895a2-cc2e-4387-acfe-8506ad6761eb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:57.431121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c962b48-41ee-48a5-9ea2-66f9fd3c4bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:57.431834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962b48-41ee-48a5-9ea2-66f9fd3c4bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:57.432119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962b48-41ee-48a5-9ea2-66f9fd3c4bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:57.432394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c962b48-41ee-48a5-9ea2-66f9fd3c4bc8 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:57.438317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c962b48-41ee-48a5-9ea2-66f9fd3c4bc8 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:57.439843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c962b48-41ee-48a5-9ea2-66f9fd3c4bc8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:57.442484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 114/230] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:57 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:57.444067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 117/231] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:15:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 13:15:57.448624 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c962b48-41ee-48a5-9ea2-66f9fd3c4bc8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:57.453720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3560b0-3e71-4862-aca1-8422337ada6d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:57.593989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:57.654891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3560b0-3e71-4862-aca1-8422337ada6d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:57.717194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0bd92-f054-446a-a88a-47e6526a34a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:57.718423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0bd92-f054-446a-a88a-47e6526a34a3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:57.718788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0bd92-f054-446a-a88a-47e6526a34a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:57.719277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0bd92-f054-446a-a88a-47e6526a34a3 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:57.720789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:57.724397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0bd92-f054-446a-a88a-47e6526a34a3 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:57.727787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0bd92-f054-446a-a88a-47e6526a34a3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:57.731394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 118/232] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:15:57 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:15:57.757642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0bd92-f054-446a-a88a-47e6526a34a3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:15:57.810674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c962b48-41ee-48a5-9ea2-66f9fd3c4bc8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:57.912549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b1827-3c6a-4ff2-a1af-6c338e1f40d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:57.913591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1827-3c6a-4ff2-a1af-6c338e1f40d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:57.914091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1827-3c6a-4ff2-a1af-6c338e1f40d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:57.914547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1827-3c6a-4ff2-a1af-6c338e1f40d5 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b/users/58c5b7efbca845bd9d5d1c8cc8f50b80/roles/753d3bbfcdaa4cef86cc8de37ae19906` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:57.918600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b1827-3c6a-4ff2-a1af-6c338e1f40d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a96496b71e840f8b6e17be47512394b, user_id=58c5b7efbca845bd9d5d1c8cc8f50b80, role_id=753d3bbfcdaa4cef86cc8de37ae19906)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:57.921500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b1827-3c6a-4ff2-a1af-6c338e1f40d5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:57.952642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 115/233] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:15:57 2022] PUT /identity/v3/projects/5a96496b71e840f8b6e17be47512394b/users/58c5b7efbca845bd9d5d1c8cc8f50b80/roles/753d3bbfcdaa4cef86cc8de37ae19906 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:15:58.007472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0bd92-f054-446a-a88a-47e6526a34a3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:15:58.073289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:15:58.090424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023e3c6-c599-4514-af4c-dbc10892452e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:15:58.091570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e3c6-c599-4514-af4c-dbc10892452e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:58.092171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e3c6-c599-4514-af4c-dbc10892452e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:58.092673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e3c6-c599-4514-af4c-dbc10892452e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:58.094046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023e3c6-c599-4514-af4c-dbc10892452e None admin] RBAC: Adding query filter params (role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=5a96496b71e840f8b6e17be47512394b, user.id=1759d68410ef41369c91d7026927f2f2) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:15:58.094387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023e3c6-c599-4514-af4c-dbc10892452e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:15:58.098387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023e3c6-c599-4514-af4c-dbc10892452e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:15:58.109788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 119/234] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:15:57 2022] GET /identity/v3/role_assignments?user.id=1759d68410ef41369c91d7026927f2f2&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 259 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:15:59.637167 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886403c-7c88-41b0-85d7-4fb47fd38601 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:59.637974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886403c-7c88-41b0-85d7-4fb47fd38601 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:59.637974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886403c-7c88-41b0-85d7-4fb47fd38601 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:59.641015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 116/235] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:59.650639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e4ec8-0bf1-498c-a6a0-3bcd0a3632ac None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:59.651413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e4ec8-0bf1-498c-a6a0-3bcd0a3632ac None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:59.651541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e4ec8-0bf1-498c-a6a0-3bcd0a3632ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:59.697299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-625e4ec8-0bf1-498c-a6a0-3bcd0a3632ac None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:59.699416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-625e4ec8-0bf1-498c-a6a0-3bcd0a3632ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:59.871422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 120/236] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:15:59.898762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14deef-f2fd-4f03-8a88-b828f2187afd None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:59.899160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14deef-f2fd-4f03-8a88-b828f2187afd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:59.899635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14deef-f2fd-4f03-8a88-b828f2187afd None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:59.902423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 117/237] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:15:59.912336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66ffe-16e9-46c1-b46e-afed411cc04a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:59.912779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66ffe-16e9-46c1-b46e-afed411cc04a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:59.913262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66ffe-16e9-46c1-b46e-afed411cc04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:59.946217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52c0cb-22bf-47c1-ab33-263ac0b04cee None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:59.946710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52c0cb-22bf-47c1-ab33-263ac0b04cee None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:59.947196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52c0cb-22bf-47c1-ab33-263ac0b04cee None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:59.949782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 118/238] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:15:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:15:59.950121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-edf66ffe-16e9-46c1-b46e-afed411cc04a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:59.952111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-edf66ffe-16e9-46c1-b46e-afed411cc04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:15:59.956049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17ca3-deb1-4c01-9ac8-faf3a3e9149a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:15:59.956543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17ca3-deb1-4c01-9ac8-faf3a3e9149a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:15:59.957232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f17ca3-deb1-4c01-9ac8-faf3a3e9149a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:15:59.993188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-86f17ca3-deb1-4c01-9ac8-faf3a3e9149a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:15:59.995722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-86f17ca3-deb1-4c01-9ac8-faf3a3e9149a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:00.134824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 121/239] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:00.208037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 119/240] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:00.451934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66ffe-16e9-46c1-b46e-afed411cc04a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:00.538697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f17ca3-deb1-4c01-9ac8-faf3a3e9149a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:00.550482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:00.653119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66ffe-16e9-46c1-b46e-afed411cc04a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:00.723180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:00.724556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf2c2c-4301-4a9e-a1e7-05b84d4015a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:00.726114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf2c2c-4301-4a9e-a1e7-05b84d4015a2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:00.726856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf2c2c-4301-4a9e-a1e7-05b84d4015a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:00.727548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf2c2c-4301-4a9e-a1e7-05b84d4015a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:00.732111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf2c2c-4301-4a9e-a1e7-05b84d4015a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:00.736478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf2c2c-4301-4a9e-a1e7-05b84d4015a2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:00.739977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 122/241] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:00.749246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf2c2c-4301-4a9e-a1e7-05b84d4015a2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:00.886604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f17ca3-deb1-4c01-9ac8-faf3a3e9149a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:00.944825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4dc6f-69bd-470f-a35f-00028d0ec6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:00.946172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4dc6f-69bd-470f-a35f-00028d0ec6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:00.946859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4dc6f-69bd-470f-a35f-00028d0ec6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:00.947287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4dc6f-69bd-470f-a35f-00028d0ec6a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:00.948925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf2c2c-4301-4a9e-a1e7-05b84d4015a2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:00.963252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4dc6f-69bd-470f-a35f-00028d0ec6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0db4dc6f-69bd-470f-a35f-00028d0ec6a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:00.969395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:00.970341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:00.971150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 120/242] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:16:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:00.972479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4dc6f-69bd-470f-a35f-00028d0ec6a7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:00.973070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:00.994344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11833eb-b198-4284-9603-260ea05ba655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:00.995422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11833eb-b198-4284-9603-260ea05ba655 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:00.995975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11833eb-b198-4284-9603-260ea05ba655 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:00.996455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11833eb-b198-4284-9603-260ea05ba655 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:00.997398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11833eb-b198-4284-9603-260ea05ba655 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:00.999202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11833eb-b198-4284-9603-260ea05ba655 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:01.027368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 123/243] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:16:00 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:16:01.029349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11833eb-b198-4284-9603-260ea05ba655 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:01.151799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4dc6f-69bd-470f-a35f-00028d0ec6a7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:01.199652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:01.256069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93ef6-af53-41d3-ba56-69b3c269320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:01.262678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93ef6-af53-41d3-ba56-69b3c269320f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:01.266651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93ef6-af53-41d3-ba56-69b3c269320f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:01.267139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93ef6-af53-41d3-ba56-69b3c269320f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:01.274646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11833eb-b198-4284-9603-260ea05ba655 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:01.295660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f93ef6-af53-41d3-ba56-69b3c269320f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-36f93ef6-af53-41d3-ba56-69b3c269320f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:01.301492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:01.302458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:01.303271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:01.305886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 121/244] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:01.312218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93ef6-af53-41d3-ba56-69b3c269320f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:01.377836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:01.380607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:01.382625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:01.383498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:01.385140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:01.386313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:01.387434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:01.412011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 124/245] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:01.422165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:01.457866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93ef6-af53-41d3-ba56-69b3c269320f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:01.494035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:01.502710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:01.504027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:01.504517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:01.504966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:01.505941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:01.506388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:01.508382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:01.519175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 122/246] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:01.525430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:01.582603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:01.652946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2a6c-1694-4be9-aba9-2039cecab77b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:01.665831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5ff07-f7e8-40b2-9003-0cb52f15cc4a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:01.726742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a08b1-36ff-49f0-b024-402b6f1c2f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:01.727996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a08b1-36ff-49f0-b024-402b6f1c2f74 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:01.728578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a08b1-36ff-49f0-b024-402b6f1c2f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:01.729145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a08b1-36ff-49f0-b024-402b6f1c2f74 None admin] PATH_INFO: `/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:01.731850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a08b1-36ff-49f0-b024-402b6f1c2f74 None admin] RBAC: Authorizing `identity:get_user(user_id=58c5b7efbca845bd9d5d1c8cc8f50b80)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:01.734895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47a08b1-36ff-49f0-b024-402b6f1c2f74 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:01.736944 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 125/247] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:01 2022] GET /identity/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80 => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:01.741000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505e230-877a-4b7d-90a7-4037b541b6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:01.742750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505e230-877a-4b7d-90a7-4037b541b6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:01.743052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505e230-877a-4b7d-90a7-4037b541b6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:01.743335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505e230-877a-4b7d-90a7-4037b541b6a5 None admin] PATH_INFO: `/v3/users/d25ddc64aaf24cd29aef2ebfe0019f67` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:01.745334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a08b1-36ff-49f0-b024-402b6f1c2f74 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:01.761346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505e230-877a-4b7d-90a7-4037b541b6a5 None admin] RBAC: Authorizing `identity:get_user(user_id=d25ddc64aaf24cd29aef2ebfe0019f67)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:01.763005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505e230-877a-4b7d-90a7-4037b541b6a5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:01.764834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 123/248] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:01 2022] GET /identity/v3/users/d25ddc64aaf24cd29aef2ebfe0019f67 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:01.771680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505e230-877a-4b7d-90a7-4037b541b6a5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:01.905087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:01.925205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505e230-877a-4b7d-90a7-4037b541b6a5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:01.953801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a08b1-36ff-49f0-b024-402b6f1c2f74 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:01.958996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:01.972762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75087b60-38fc-483a-9b76-ac928ecf22a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:01.974558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75087b60-38fc-483a-9b76-ac928ecf22a5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:01.974822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75087b60-38fc-483a-9b76-ac928ecf22a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:01.975106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75087b60-38fc-483a-9b76-ac928ecf22a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.013240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b844e0d-0948-470f-ad77-69d81299fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:02.014056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b844e0d-0948-470f-ad77-69d81299fe07 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:02.014339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b844e0d-0948-470f-ad77-69d81299fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:02.014614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b844e0d-0948-470f-ad77-69d81299fe07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.097061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75087b60-38fc-483a-9b76-ac928ecf22a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:02.099439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75087b60-38fc-483a-9b76-ac928ecf22a5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:02.130600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b844e0d-0948-470f-ad77-69d81299fe07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:02.133825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b844e0d-0948-470f-ad77-69d81299fe07 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:02.221182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:02.231604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 124/249] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:02.233340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626c5d-4a18-4e01-99da-21aa51bf19bc None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:02.233875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626c5d-4a18-4e01-99da-21aa51bf19bc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:02.234333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626c5d-4a18-4e01-99da-21aa51bf19bc None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.236295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 125/250] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:02.238689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd213d-a7b5-4d55-851a-c4d5d59bfd0e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:02.239375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd213d-a7b5-4d55-851a-c4d5d59bfd0e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:02.239832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd213d-a7b5-4d55-851a-c4d5d59bfd0e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.241500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 126/251] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:02.244825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef132a0-684f-4692-92ef-bb8e18dd2a64 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:02.245336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef132a0-684f-4692-92ef-bb8e18dd2a64 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:02.245829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef132a0-684f-4692-92ef-bb8e18dd2a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.269111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6ef132a0-684f-4692-92ef-bb8e18dd2a64 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:02.270801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6ef132a0-684f-4692-92ef-bb8e18dd2a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:02.353938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 126/252] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:02.354538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b844e0d-0948-470f-ad77-69d81299fe07 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:02.363113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:02.379504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 127/253] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:02.380888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64171482-9bfa-429d-a8e7-9fee9923e129 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:02.381224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64171482-9bfa-429d-a8e7-9fee9923e129 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:02.381527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64171482-9bfa-429d-a8e7-9fee9923e129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.404862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-64171482-9bfa-429d-a8e7-9fee9923e129 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:02.406149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-64171482-9bfa-429d-a8e7-9fee9923e129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:02.520835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 128/254] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:02.521672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64171482-9bfa-429d-a8e7-9fee9923e129 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:02.600340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:02.644170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b844e0d-0948-470f-ad77-69d81299fe07 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:02.702529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64171482-9bfa-429d-a8e7-9fee9923e129 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:02.731293 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73106013-377e-413f-90eb-0b388864591d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:02.732065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73106013-377e-413f-90eb-0b388864591d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:02.732430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73106013-377e-413f-90eb-0b388864591d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:02.732606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73106013-377e-413f-90eb-0b388864591d None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.734019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73106013-377e-413f-90eb-0b388864591d None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:02.735581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73106013-377e-413f-90eb-0b388864591d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:02.737742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 127/255] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:02 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:02.749101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73106013-377e-413f-90eb-0b388864591d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:02.779879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca9128-b218-4aca-99e5-cdfcb4713644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:02.780663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9128-b218-4aca-99e5-cdfcb4713644 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:02.780990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9128-b218-4aca-99e5-cdfcb4713644 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:02.781262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca9128-b218-4aca-99e5-cdfcb4713644 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.782520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca9128-b218-4aca-99e5-cdfcb4713644 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:02.784167 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca9128-b218-4aca-99e5-cdfcb4713644 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:02.786239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 129/256] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:02 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:02.793240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca9128-b218-4aca-99e5-cdfcb4713644 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:02.890248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:02.928175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73106013-377e-413f-90eb-0b388864591d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:02.982221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:02.983141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:02.983578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:02.985990 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:02.987866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] RBAC: Adding query filter params (role.id=cd3aa49828434199911a5124c59be2a4, scope.project.id=5a96496b71e840f8b6e17be47512394b, user.id=d25ddc64aaf24cd29aef2ebfe0019f67) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:02.988540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:02.990356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:03.006820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 128/257] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:02 2022] GET /identity/v3/role_assignments?user.id=d25ddc64aaf24cd29aef2ebfe0019f67&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=cd3aa49828434199911a5124c59be2a4 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:03.041419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca9128-b218-4aca-99e5-cdfcb4713644 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:03.066774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:03.110635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:03.137780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:03.138771 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:03.139191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:03.139602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:03.140486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=5a96496b71e840f8b6e17be47512394b, user.id=58c5b7efbca845bd9d5d1c8cc8f50b80) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:03.140971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:03.142622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:03.161192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 130/258] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:02 2022] GET /identity/v3/role_assignments?user.id=58c5b7efbca845bd9d5d1c8cc8f50b80&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 608 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:03.306071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73707da6-c3d4-4aee-972a-0da92f37a3a0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:03.316138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:03.362096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9ddff-c868-4a14-bf72-7cc3bbdeb3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:03.363455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ddff-c868-4a14-bf72-7cc3bbdeb3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:03.363455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ddff-c868-4a14-bf72-7cc3bbdeb3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:03.363769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9ddff-c868-4a14-bf72-7cc3bbdeb3d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:03.367011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:03.385370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9ddff-c868-4a14-bf72-7cc3bbdeb3d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-94e9ddff-c868-4a14-bf72-7cc3bbdeb3d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:03.396056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:03.398337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:03.400198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 129/259] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:03.405487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9ddff-c868-4a14-bf72-7cc3bbdeb3d1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:03.539493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9ddff-c868-4a14-bf72-7cc3bbdeb3d1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:03.577929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20637dda-1680-4456-895d-26fd68b7e3df None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:03.595393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:03.596084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:03.596479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:03.596807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:03.597853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:03.598173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:03.599656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:03.615790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 130/260] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:03.618141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:03.623782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecff55f-ca75-4bb3-90dc-3e9fd9712b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:03.624426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecff55f-ca75-4bb3-90dc-3e9fd9712b52 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:03.624699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecff55f-ca75-4bb3-90dc-3e9fd9712b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:03.624961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecff55f-ca75-4bb3-90dc-3e9fd9712b52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:03.649795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecff55f-ca75-4bb3-90dc-3e9fd9712b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0ecff55f-ca75-4bb3-90dc-3e9fd9712b52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:03.658387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:03.660757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:03.664011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 131/261] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:16:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:03.672477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecff55f-ca75-4bb3-90dc-3e9fd9712b52 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:03.816209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:03.876117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:03.925585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591507c-470d-4ce8-bb22-862ee838fb93 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:03.982124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecff55f-ca75-4bb3-90dc-3e9fd9712b52 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:03.996515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b6dac-502d-4535-990e-687c310bee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:03.997184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b6dac-502d-4535-990e-687c310bee01 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:03.997493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b6dac-502d-4535-990e-687c310bee01 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:03.997807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b6dac-502d-4535-990e-687c310bee01 None admin] PATH_INFO: `/v3/users/1759d68410ef41369c91d7026927f2f2` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:04.000769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b6dac-502d-4535-990e-687c310bee01 None admin] RBAC: Authorizing `identity:get_user(user_id=1759d68410ef41369c91d7026927f2f2)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:04.002271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b6dac-502d-4535-990e-687c310bee01 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:04.004194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 131/262] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:03 2022] GET /identity/v3/users/1759d68410ef41369c91d7026927f2f2 => generated 297 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:04.011420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b6dac-502d-4535-990e-687c310bee01 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:04.062298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:04.062945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:04.063228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:04.063539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:04.064192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:04.064498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:04.066003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:04.081413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 132/263] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:04.090420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:04.236535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b6dac-502d-4535-990e-687c310bee01 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:04.354972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:04.378825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118756f-2204-4881-9d12-ad493efd09d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:04.379686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118756f-2204-4881-9d12-ad493efd09d8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:04.380072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118756f-2204-4881-9d12-ad493efd09d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:04.380452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118756f-2204-4881-9d12-ad493efd09d8 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:04.382171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118756f-2204-4881-9d12-ad493efd09d8 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:04.383773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118756f-2204-4881-9d12-ad493efd09d8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:04.385833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 132/264] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:04 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:04.391764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118756f-2204-4881-9d12-ad493efd09d8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:04.394772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b41c4-2c39-46c3-a623-3b0085e0a039 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:04.415510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:04.462185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef8b45-1c0a-4a1b-b903-d4ba6b2f119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:04.463299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef8b45-1c0a-4a1b-b903-d4ba6b2f119f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:04.463799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef8b45-1c0a-4a1b-b903-d4ba6b2f119f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:04.464275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef8b45-1c0a-4a1b-b903-d4ba6b2f119f None admin] PATH_INFO: `/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:04.466575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef8b45-1c0a-4a1b-b903-d4ba6b2f119f None admin] RBAC: Authorizing `identity:get_user(user_id=58c5b7efbca845bd9d5d1c8cc8f50b80)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:04.469627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef8b45-1c0a-4a1b-b903-d4ba6b2f119f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:04.473354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 133/265] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:04 2022] GET /identity/v3/users/58c5b7efbca845bd9d5d1c8cc8f50b80 => generated 283 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:04.482920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef8b45-1c0a-4a1b-b903-d4ba6b2f119f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:04.595936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118756f-2204-4881-9d12-ad493efd09d8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:04.670797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765de6-876f-4cd7-811d-31557de416c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:04.671519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765de6-876f-4cd7-811d-31557de416c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:04.671812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765de6-876f-4cd7-811d-31557de416c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:04.672098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765de6-876f-4cd7-811d-31557de416c1 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b/users/1759d68410ef41369c91d7026927f2f2/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:04.678979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765de6-876f-4cd7-811d-31557de416c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a96496b71e840f8b6e17be47512394b, user_id=1759d68410ef41369c91d7026927f2f2, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:04.680486 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765de6-876f-4cd7-811d-31557de416c1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:04.688628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef8b45-1c0a-4a1b-b903-d4ba6b2f119f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:04.708521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 133/266] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:16:04 2022] PUT /identity/v3/projects/5a96496b71e840f8b6e17be47512394b/users/1759d68410ef41369c91d7026927f2f2/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:16:04.759063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c2834-1211-441b-b464-4de814a0712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:04.760117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c2834-1211-441b-b464-4de814a0712c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:04.760619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c2834-1211-441b-b464-4de814a0712c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:04.761183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c2834-1211-441b-b464-4de814a0712c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:04.859399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:04.907706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c2834-1211-441b-b464-4de814a0712c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:04.911406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0c2834-1211-441b-b464-4de814a0712c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:04.955060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251a96e-195a-456a-96f8-025c387dd1a0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:04.955990 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251a96e-195a-456a-96f8-025c387dd1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:04.956717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251a96e-195a-456a-96f8-025c387dd1a0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:04.959929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 134/267] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:04.966956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef1a3b-e564-4173-a28a-cb6ba44db167 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:04.967473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef1a3b-e564-4173-a28a-cb6ba44db167 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:04.967900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef1a3b-e564-4173-a28a-cb6ba44db167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:04.997211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-54ef1a3b-e564-4173-a28a-cb6ba44db167 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:04.999422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-54ef1a3b-e564-4173-a28a-cb6ba44db167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:05.092622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:05.116368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 134/268] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:05.125628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c2834-1211-441b-b464-4de814a0712c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:05.138866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 135/269] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:05.279241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:05.335296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c2834-1211-441b-b464-4de814a0712c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:05.407306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26027636-4b90-4dfa-b8b2-a641ad76b8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:05.409600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26027636-4b90-4dfa-b8b2-a641ad76b8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:05.410273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26027636-4b90-4dfa-b8b2-a641ad76b8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:05.410980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26027636-4b90-4dfa-b8b2-a641ad76b8d4 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:05.413270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26027636-4b90-4dfa-b8b2-a641ad76b8d4 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:05.416220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26027636-4b90-4dfa-b8b2-a641ad76b8d4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:05.419702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 135/270] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:05 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:05.439547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef1a3b-e564-4173-a28a-cb6ba44db167 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:05.672869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef1a3b-e564-4173-a28a-cb6ba44db167 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:05.711392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08cc09-7db8-470a-b034-1c599c63c26d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:05.712042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08cc09-7db8-470a-b034-1c599c63c26d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:05.712527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08cc09-7db8-470a-b034-1c599c63c26d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:05.714542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 136/271] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:05.724840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08cc09-7db8-470a-b034-1c599c63c26d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:05.741288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa0a28-bc80-4127-832c-40dffdcdb636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:05.742380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa0a28-bc80-4127-832c-40dffdcdb636 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:05.742884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa0a28-bc80-4127-832c-40dffdcdb636 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:05.743380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa0a28-bc80-4127-832c-40dffdcdb636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:05.744825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa0a28-bc80-4127-832c-40dffdcdb636 None admin] RBAC: Adding query filter params (role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=5a96496b71e840f8b6e17be47512394b, user.id=58c5b7efbca845bd9d5d1c8cc8f50b80) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:05.745390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa0a28-bc80-4127-832c-40dffdcdb636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:05.748198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa0a28-bc80-4127-832c-40dffdcdb636 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:05.760716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 136/272] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:05 2022] GET /identity/v3/role_assignments?user.id=58c5b7efbca845bd9d5d1c8cc8f50b80&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:05.763532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8b59c-f504-42f8-836f-349b32726e28 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:05.764280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8b59c-f504-42f8-836f-349b32726e28 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:05.765110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8b59c-f504-42f8-836f-349b32726e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:05.814522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-81c8b59c-f504-42f8-836f-349b32726e28 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:05.817930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-81c8b59c-f504-42f8-836f-349b32726e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:05.978469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08cc09-7db8-470a-b034-1c599c63c26d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:05.985839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:06.053348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492fd48-f60d-46bb-8853-edb3e1fb0429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:06.054884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492fd48-f60d-46bb-8853-edb3e1fb0429 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:06.055387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492fd48-f60d-46bb-8853-edb3e1fb0429 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:06.055883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492fd48-f60d-46bb-8853-edb3e1fb0429 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:06.080858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492fd48-f60d-46bb-8853-edb3e1fb0429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:06.084315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 137/273] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c492fd48-f60d-46bb-8853-edb3e1fb0429 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:06.090706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:06.092350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:06.094403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 137/274] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:06.102690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8b59c-f504-42f8-836f-349b32726e28 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:06.303903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8b59c-f504-42f8-836f-349b32726e28 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:06.370770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef42a-36dd-4157-ad2e-c4679af5ffda None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:06.371269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef42a-36dd-4157-ad2e-c4679af5ffda None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:06.371744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef42a-36dd-4157-ad2e-c4679af5ffda None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:06.374399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 138/275] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:06.376922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:06.378673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:06.379171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:06.379655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:06.382082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:06.382733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:06.383427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e24d0-7a7f-41d3-833b-14cc2a32ec08 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:06.383914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e24d0-7a7f-41d3-833b-14cc2a32ec08 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:06.384399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e24d0-7a7f-41d3-833b-14cc2a32ec08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:06.385127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:06.396601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 138/276] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:06.404745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:06.416187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:06.427898 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4c8e24d0-7a7f-41d3-833b-14cc2a32ec08 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:06.431392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4c8e24d0-7a7f-41d3-833b-14cc2a32ec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:06.467672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:06.573327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca34a8-16a3-4a7e-968d-f2cd7da80db5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:06.615321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 139/277] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:06.634507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69658906-b936-47d9-9c0d-361c6a808df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:06.636067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658906-b936-47d9-9c0d-361c6a808df4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:06.636864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658906-b936-47d9-9c0d-361c6a808df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:06.637721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69658906-b936-47d9-9c0d-361c6a808df4 None admin] PATH_INFO: `/v3/users/10c1b3f9599043a68598e07801183dee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:06.665439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69658906-b936-47d9-9c0d-361c6a808df4 None admin] RBAC: Authorizing `identity:get_user(user_id=10c1b3f9599043a68598e07801183dee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:06.668763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69658906-b936-47d9-9c0d-361c6a808df4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:06.672593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 139/278] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:06 2022] GET /identity/v3/users/10c1b3f9599043a68598e07801183dee => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:06.678335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e24d0-7a7f-41d3-833b-14cc2a32ec08 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:06.727976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69658906-b936-47d9-9c0d-361c6a808df4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:06.844770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e24d0-7a7f-41d3-833b-14cc2a32ec08 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:06.910566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:06.911735 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:06.912231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:06.912706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:06.914704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] RBAC: Adding query filter params (role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, user.id=10c1b3f9599043a68598e07801183dee, scope.system=all) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:06.914883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:06.917747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:06.923269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:06.929240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:06.937920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 140/279] 10.222.0.58 () {56 vars in 1308 bytes} [Mon Oct 3 13:16:06 2022] GET /identity/v3/role_assignments?user.id=10c1b3f9599043a68598e07801183dee&scope.system=all&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:06.942396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69658906-b936-47d9-9c0d-361c6a808df4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:06.950860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:07.036044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866bf2c-16c1-4528-a836-80b878034aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:07.037351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866bf2c-16c1-4528-a836-80b878034aef None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:07.038062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866bf2c-16c1-4528-a836-80b878034aef None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:07.038576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866bf2c-16c1-4528-a836-80b878034aef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:07.040645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866bf2c-16c1-4528-a836-80b878034aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:07.043953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866bf2c-16c1-4528-a836-80b878034aef None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:07.052364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 140/280] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:07.055167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866bf2c-16c1-4528-a836-80b878034aef None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:07.109260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15912f9-7b52-4eec-95f6-1d0e9254d3e0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:07.167206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a026-7b5c-4a9d-82ba-8fd5f8895622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:07.168530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a026-7b5c-4a9d-82ba-8fd5f8895622 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:07.169171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a026-7b5c-4a9d-82ba-8fd5f8895622 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:07.170033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a026-7b5c-4a9d-82ba-8fd5f8895622 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:07.191987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a026-7b5c-4a9d-82ba-8fd5f8895622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-9b58a026-7b5c-4a9d-82ba-8fd5f8895622 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:07.203202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:07.204346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:07.205203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:07.209526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 141/281] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:07.214178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a026-7b5c-4a9d-82ba-8fd5f8895622 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:07.235377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866bf2c-16c1-4528-a836-80b878034aef None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:07.291300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6886b43-b0a7-48ee-a64b-323f86c717a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:07.292342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6886b43-b0a7-48ee-a64b-323f86c717a7 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:07.292827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6886b43-b0a7-48ee-a64b-323f86c717a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:07.293323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6886b43-b0a7-48ee-a64b-323f86c717a7 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:07.294611 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6886b43-b0a7-48ee-a64b-323f86c717a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:07.297432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6886b43-b0a7-48ee-a64b-323f86c717a7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:07.306121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:07.345481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 141/282] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:16:07 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:16:07.388982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a026-7b5c-4a9d-82ba-8fd5f8895622 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:07.462450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b7b60-cf7d-451c-9b35-cd7dc8075a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:07.463617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b7b60-cf7d-451c-9b35-cd7dc8075a08 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:07.464134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b7b60-cf7d-451c-9b35-cd7dc8075a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:07.464623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b7b60-cf7d-451c-9b35-cd7dc8075a08 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:07.465954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b7b60-cf7d-451c-9b35-cd7dc8075a08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:07.466496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b7b60-cf7d-451c-9b35-cd7dc8075a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:07.469304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b7b60-cf7d-451c-9b35-cd7dc8075a08 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:07.488284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 142/283] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:07.498522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6886b43-b0a7-48ee-a64b-323f86c717a7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:07.553650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:07.706719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6886b43-b0a7-48ee-a64b-323f86c717a7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:07.750644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fac7d-ecc4-4a32-ae24-2087fcdaaf9c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:07.750992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fac7d-ecc4-4a32-ae24-2087fcdaaf9c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:07.751434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154fac7d-ecc4-4a32-ae24-2087fcdaaf9c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:07.753948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ded94-1a76-4e7f-a928-78c7ec19f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:07.754623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ded94-1a76-4e7f-a928-78c7ec19f44e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:07.754915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ded94-1a76-4e7f-a928-78c7ec19f44e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:07.755202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ded94-1a76-4e7f-a928-78c7ec19f44e None admin] PATH_INFO: `/v3/users/d25ddc64aaf24cd29aef2ebfe0019f67` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:07.757075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 143/284] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:16:07.759210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ded94-1a76-4e7f-a928-78c7ec19f44e None admin] RBAC: Authorizing `identity:get_user(user_id=d25ddc64aaf24cd29aef2ebfe0019f67)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:07.760765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ded94-1a76-4e7f-a928-78c7ec19f44e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:07.762893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 142/285] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:07 2022] GET /identity/v3/users/d25ddc64aaf24cd29aef2ebfe0019f67 => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:07.765989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e767-5c90-4aaa-a8f9-67364b65fbdc None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:07.766447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e767-5c90-4aaa-a8f9-67364b65fbdc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:07.766858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e767-5c90-4aaa-a8f9-67364b65fbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:07.771883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ded94-1a76-4e7f-a928-78c7ec19f44e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:07.804505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5f43e767-5c90-4aaa-a8f9-67364b65fbdc None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:07.810314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5f43e767-5c90-4aaa-a8f9-67364b65fbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:07.882538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:07.907037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ded94-1a76-4e7f-a928-78c7ec19f44e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:07.967334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 144/286] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:07.975565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902c63-82df-4e5e-8fa0-55c72d08e9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:07.976608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902c63-82df-4e5e-8fa0-55c72d08e9de None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:07.977066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902c63-82df-4e5e-8fa0-55c72d08e9de None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:07.977672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902c63-82df-4e5e-8fa0-55c72d08e9de None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:07.979745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902c63-82df-4e5e-8fa0-55c72d08e9de None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:07.982413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902c63-82df-4e5e-8fa0-55c72d08e9de None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:07.986185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 143/287] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:07 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:07.994981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43e767-5c90-4aaa-a8f9-67364b65fbdc None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:08.065943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:08.120346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43e767-5c90-4aaa-a8f9-67364b65fbdc None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:08.160180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1f419-f567-444f-8d0f-650324689123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:08.160895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1f419-f567-444f-8d0f-650324689123 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:08.161183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1f419-f567-444f-8d0f-650324689123 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:08.161492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1f419-f567-444f-8d0f-650324689123 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b/users/d25ddc64aaf24cd29aef2ebfe0019f67/roles/cd3aa49828434199911a5124c59be2a4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:08.163627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1f419-f567-444f-8d0f-650324689123 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a96496b71e840f8b6e17be47512394b, user_id=d25ddc64aaf24cd29aef2ebfe0019f67, role_id=cd3aa49828434199911a5124c59be2a4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:08.165113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc1f419-f567-444f-8d0f-650324689123 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:08.181607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 145/288] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:16:07 2022] PUT /identity/v3/projects/5a96496b71e840f8b6e17be47512394b/users/d25ddc64aaf24cd29aef2ebfe0019f67/roles/cd3aa49828434199911a5124c59be2a4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:16:08.256952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902c63-82df-4e5e-8fa0-55c72d08e9de None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:08.437036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902c63-82df-4e5e-8fa0-55c72d08e9de None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:08.491083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da45ac-8fb0-4b7c-acae-41f32f5b2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:08.491766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da45ac-8fb0-4b7c-acae-41f32f5b2ffb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:08.492058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da45ac-8fb0-4b7c-acae-41f32f5b2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:08.492343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da45ac-8fb0-4b7c-acae-41f32f5b2ffb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:08.510693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da45ac-8fb0-4b7c-acae-41f32f5b2ffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-20da45ac-8fb0-4b7c-acae-41f32f5b2ffb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:08.516263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:08.517242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:08.518641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 144/289] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:08.527139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1f419-f567-444f-8d0f-650324689123 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:08.655790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1f419-f567-444f-8d0f-650324689123 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:08.679323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:08.709198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:08.709882 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:08.710160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:08.710440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:08.711148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:08.711460 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:08.712998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:08.724179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 146/290] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:08.734592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da45ac-8fb0-4b7c-acae-41f32f5b2ffb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:08.767404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:08.903983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da45ac-8fb0-4b7c-acae-41f32f5b2ffb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:08.955864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77352f0-4e1d-4f1c-843b-5cb63a823b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:08.956931 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77352f0-4e1d-4f1c-843b-5cb63a823b0e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:08.957378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77352f0-4e1d-4f1c-843b-5cb63a823b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:08.957840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77352f0-4e1d-4f1c-843b-5cb63a823b0e None admin] PATH_INFO: `/v3/users/1759d68410ef41369c91d7026927f2f2` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:08.961593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77352f0-4e1d-4f1c-843b-5cb63a823b0e None admin] RBAC: Authorizing `identity:get_user(user_id=1759d68410ef41369c91d7026927f2f2)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:08.963346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77352f0-4e1d-4f1c-843b-5cb63a823b0e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:08.965646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 145/291] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:08 2022] GET /identity/v3/users/1759d68410ef41369c91d7026927f2f2 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:08.972198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:09.179962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bf62e-451e-4a0e-8781-c425ef1ca8a5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:09.241111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71cdcd-1079-41d0-ba2d-6369ff5c7fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:09.241792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cdcd-1079-41d0-ba2d-6369ff5c7fcc None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:09.242080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cdcd-1079-41d0-ba2d-6369ff5c7fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:09.242361 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cdcd-1079-41d0-ba2d-6369ff5c7fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:09.316198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:09.371706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71cdcd-1079-41d0-ba2d-6369ff5c7fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:09.373289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71cdcd-1079-41d0-ba2d-6369ff5c7fcc None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:09.541352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 147/292] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:09.550140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77352f0-4e1d-4f1c-843b-5cb63a823b0e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:09.713834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:09.739551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77352f0-4e1d-4f1c-843b-5cb63a823b0e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:09.802155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53424a4-de26-4c89-b08d-efd4db2d9503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:09.803161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53424a4-de26-4c89-b08d-efd4db2d9503 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:09.803671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53424a4-de26-4c89-b08d-efd4db2d9503 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:09.804164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53424a4-de26-4c89-b08d-efd4db2d9503 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:09.806272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53424a4-de26-4c89-b08d-efd4db2d9503 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:09.808377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53424a4-de26-4c89-b08d-efd4db2d9503 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:09.810147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 146/293] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:09 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:09.820405 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71cdcd-1079-41d0-ba2d-6369ff5c7fcc None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:09.957181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:10.000484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71cdcd-1079-41d0-ba2d-6369ff5c7fcc None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:10.055786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225b1fb-9fdc-4b37-9986-96403ab796e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:10.056780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b1fb-9fdc-4b37-9986-96403ab796e7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:10.057265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b1fb-9fdc-4b37-9986-96403ab796e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:10.057845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b1fb-9fdc-4b37-9986-96403ab796e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:10.058797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225b1fb-9fdc-4b37-9986-96403ab796e7 None admin] RBAC: Adding query filter params (role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=5a96496b71e840f8b6e17be47512394b, user.id=1759d68410ef41369c91d7026927f2f2) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:10.059306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225b1fb-9fdc-4b37-9986-96403ab796e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:10.061011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225b1fb-9fdc-4b37-9986-96403ab796e7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:10.068753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 148/294] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:09 2022] GET /identity/v3/role_assignments?user.id=1759d68410ef41369c91d7026927f2f2&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:10.433225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeab41-9e74-4eb3-9c74-1d818e5b0a49 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:10.434495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeab41-9e74-4eb3-9c74-1d818e5b0a49 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:10.434495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efeab41-9e74-4eb3-9c74-1d818e5b0a49 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:10.436179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 147/295] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:10.456344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794600db-d23e-495c-bdb2-fc1b821a5876 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:10.457822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794600db-d23e-495c-bdb2-fc1b821a5876 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:10.458548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794600db-d23e-495c-bdb2-fc1b821a5876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:10.490098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-794600db-d23e-495c-bdb2-fc1b821a5876 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:10.492556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-794600db-d23e-495c-bdb2-fc1b821a5876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:10.586196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 149/296] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:10.789109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecdad-71c1-4265-90d4-8933be02687f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:10.790129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecdad-71c1-4265-90d4-8933be02687f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:10.790661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacecdad-71c1-4265-90d4-8933be02687f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:10.792908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 148/297] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:10.799539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6c1f3-3f20-40e9-8bfa-2382ffc841a0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:10.799972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6c1f3-3f20-40e9-8bfa-2382ffc841a0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:10.800266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6c1f3-3f20-40e9-8bfa-2382ffc841a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:10.825007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-15b6c1f3-3f20-40e9-8bfa-2382ffc841a0 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:10.826344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-15b6c1f3-3f20-40e9-8bfa-2382ffc841a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:10.904767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:10.912228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 150/298] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:10.920355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc95c9-fa2c-4d92-b7a9-a1469e4b0b5d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:10.920986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc95c9-fa2c-4d92-b7a9-a1469e4b0b5d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:10.921537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc95c9-fa2c-4d92-b7a9-a1469e4b0b5d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:10.923347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 149/299] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:10.929153 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf872b2a-2594-4527-bc23-8a2191ef628b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:10.929613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf872b2a-2594-4527-bc23-8a2191ef628b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:10.929781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf872b2a-2594-4527-bc23-8a2191ef628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:10.942338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc95c9-fa2c-4d92-b7a9-a1469e4b0b5d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:10.951135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-bf872b2a-2594-4527-bc23-8a2191ef628b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:10.952363 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bf872b2a-2594-4527-bc23-8a2191ef628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:11.046433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 151/300] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:11.074709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc95c9-fa2c-4d92-b7a9-a1469e4b0b5d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:11.142646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62e6cc-2fd3-467d-8c86-e3e7261335db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:11.144168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62e6cc-2fd3-467d-8c86-e3e7261335db None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.144938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62e6cc-2fd3-467d-8c86-e3e7261335db None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.145746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62e6cc-2fd3-467d-8c86-e3e7261335db None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.162331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62e6cc-2fd3-467d-8c86-e3e7261335db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9c62e6cc-2fd3-467d-8c86-e3e7261335db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:11.167585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:11.168463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:11.169264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:11.171649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 150/301] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:11.178591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf872b2a-2594-4527-bc23-8a2191ef628b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:11.197238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62e6cc-2fd3-467d-8c86-e3e7261335db None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:11.233562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:11.341400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf872b2a-2594-4527-bc23-8a2191ef628b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:11.386044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:11.396716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:11.397515 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.397802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.398083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.398797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:11.399105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:11.400575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:11.414700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62e6cc-2fd3-467d-8c86-e3e7261335db None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:11.415944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 152/302] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:11.417202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:11.469999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dcbc2-82d7-4930-a444-7073a4a53c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:11.470862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dcbc2-82d7-4930-a444-7073a4a53c5c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.471008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dcbc2-82d7-4930-a444-7073a4a53c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.471345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dcbc2-82d7-4930-a444-7073a4a53c5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.472846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228dcbc2-82d7-4930-a444-7073a4a53c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:11.474637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228dcbc2-82d7-4930-a444-7073a4a53c5c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:11.476802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 151/303] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:11.479005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dcbc2-82d7-4930-a444-7073a4a53c5c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:11.591976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8e25c-f8e8-4175-b39c-ff07ad2bdf5e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:11.639697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:11.640320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44572c-c10a-4c33-a305-8dfcf049701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:11.641048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44572c-c10a-4c33-a305-8dfcf049701c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.641335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44572c-c10a-4c33-a305-8dfcf049701c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.641647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44572c-c10a-4c33-a305-8dfcf049701c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.656969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44572c-c10a-4c33-a305-8dfcf049701c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-bd44572c-c10a-4c33-a305-8dfcf049701c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:11.662733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:11.664188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 153/304] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:11.666772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44572c-c10a-4c33-a305-8dfcf049701c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:11.672506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dcbc2-82d7-4930-a444-7073a4a53c5c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:11.721302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e683586-3e06-4e97-adc1-ec187995dd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:11.721966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e683586-3e06-4e97-adc1-ec187995dd46 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.722433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e683586-3e06-4e97-adc1-ec187995dd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.722727 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e683586-3e06-4e97-adc1-ec187995dd46 None admin] PATH_INFO: `/v3/users/10c1b3f9599043a68598e07801183dee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.724089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e683586-3e06-4e97-adc1-ec187995dd46 None admin] RBAC: Authorizing `identity:get_user(user_id=10c1b3f9599043a68598e07801183dee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:11.730395 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e683586-3e06-4e97-adc1-ec187995dd46 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:11.732154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 152/305] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:11 2022] GET /identity/v3/users/10c1b3f9599043a68598e07801183dee => generated 293 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:11.732980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e683586-3e06-4e97-adc1-ec187995dd46 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:11.747208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:11.820627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44572c-c10a-4c33-a305-8dfcf049701c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:11.860724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbb450-4472-4f39-94c3-31a5da6e4a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:11.861353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbb450-4472-4f39-94c3-31a5da6e4a60 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.861711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbb450-4472-4f39-94c3-31a5da6e4a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.861994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbb450-4472-4f39-94c3-31a5da6e4a60 None admin] PATH_INFO: `/v3/groups` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.862649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbb450-4472-4f39-94c3-31a5da6e4a60 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:11.864135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbb450-4472-4f39-94c3-31a5da6e4a60 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:11.866308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e683586-3e06-4e97-adc1-ec187995dd46 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:11.884372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 154/306] 10.222.0.58 () {60 vars in 1134 bytes} [Mon Oct 3 13:16:11 2022] POST /identity/v3/groups => generated 229 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:16:11.891429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11a8c-4496-4688-a7a9-43579f013d8d None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.892834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11a8c-4496-4688-a7a9-43579f013d8d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.893836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f11a8c-4496-4688-a7a9-43579f013d8d None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.895664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 155/307] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:11.896372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11a8c-4496-4688-a7a9-43579f013d8d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:11.918080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f232e-57bb-4482-8ae6-ed157f0f9c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:11.918643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f232e-57bb-4482-8ae6-ed157f0f9c07 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.918936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f232e-57bb-4482-8ae6-ed157f0f9c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.919207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f232e-57bb-4482-8ae6-ed157f0f9c07 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.919918 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f232e-57bb-4482-8ae6-ed157f0f9c07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:11.920233 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f232e-57bb-4482-8ae6-ed157f0f9c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:11.921761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f232e-57bb-4482-8ae6-ed157f0f9c07 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:11.937059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 153/308] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:11.938637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f24d0-b085-4c40-b650-21ce5aeb00df None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:11.938946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f24d0-b085-4c40-b650-21ce5aeb00df None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:11.939221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f24d0-b085-4c40-b650-21ce5aeb00df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:11.966226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d16f24d0-b085-4c40-b650-21ce5aeb00df None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:11.967854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d16f24d0-b085-4c40-b650-21ce5aeb00df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:11.978418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:12.051496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f11a8c-4496-4688-a7a9-43579f013d8d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:12.064080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:12.064323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 154/309] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:12.065489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f24d0-b085-4c40-b650-21ce5aeb00df None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:12.098876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61fc3d-625c-48d3-a479-3847d578807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:12.099781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61fc3d-625c-48d3-a479-3847d578807a None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:12.100152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61fc3d-625c-48d3-a479-3847d578807a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:12.100504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61fc3d-625c-48d3-a479-3847d578807a None admin] PATH_INFO: `/v3/system/users/10c1b3f9599043a68598e07801183dee/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:12.103055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61fc3d-625c-48d3-a479-3847d578807a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=10c1b3f9599043a68598e07801183dee, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:12.104573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61fc3d-625c-48d3-a479-3847d578807a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:12.116004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 156/310] 10.222.0.58 () {58 vars in 1334 bytes} [Mon Oct 3 13:16:11 2022] PUT /identity/v3/system/users/10c1b3f9599043a68598e07801183dee/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:16:12.213672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f24d0-b085-4c40-b650-21ce5aeb00df None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:12.259101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123e705-bbee-4f1f-8429-a141d6a495c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:12.259834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123e705-bbee-4f1f-8429-a141d6a495c3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:12.260125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123e705-bbee-4f1f-8429-a141d6a495c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:12.260407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123e705-bbee-4f1f-8429-a141d6a495c3 None admin] PATH_INFO: `/v3/users/46c98e88c1bf474895eda8f950ca4f5c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:12.276541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:12.280851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123e705-bbee-4f1f-8429-a141d6a495c3 None admin] RBAC: Authorizing `identity:get_user(user_id=46c98e88c1bf474895eda8f950ca4f5c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:12.282575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123e705-bbee-4f1f-8429-a141d6a495c3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:12.284850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 155/311] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:12 2022] GET /identity/v3/users/46c98e88c1bf474895eda8f950ca4f5c => generated 293 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:12.291360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61fc3d-625c-48d3-a479-3847d578807a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:12.384410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123e705-bbee-4f1f-8429-a141d6a495c3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:12.416740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61fc3d-625c-48d3-a479-3847d578807a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:12.455082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:12.455753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:12.456041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:12.456319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:12.457069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] RBAC: Adding query filter params (role.id=cd3aa49828434199911a5124c59be2a4, user.id=46c98e88c1bf474895eda8f950ca4f5c, scope.system=all) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:12.457384 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:12.459000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:12.465092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 157/312] 10.222.0.58 () {56 vars in 1308 bytes} [Mon Oct 3 13:16:12 2022] GET /identity/v3/role_assignments?user.id=46c98e88c1bf474895eda8f950ca4f5c&scope.system=all&role.id=cd3aa49828434199911a5124c59be2a4 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:12.513346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123e705-bbee-4f1f-8429-a141d6a495c3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:12.565064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9367a2-a961-4f17-89b7-54cf99d1e5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:12.565771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9367a2-a961-4f17-89b7-54cf99d1e5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:12.566051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9367a2-a961-4f17-89b7-54cf99d1e5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:12.566327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9367a2-a961-4f17-89b7-54cf99d1e5a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:12.579345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9367a2-a961-4f17-89b7-54cf99d1e5a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5d9367a2-a961-4f17-89b7-54cf99d1e5a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:12.584859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:12.587110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:12.589125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 156/313] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:12.592554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:12.660430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:12.748068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7346af9f-a035-49a5-8df2-379ceb374020 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:12.790990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:12.791628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:12.791910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:12.792186 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:12.792875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:12.793177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:12.794750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:12.805372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 158/314] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:12.815466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9367a2-a961-4f17-89b7-54cf99d1e5a3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:12.907130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:12.974803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9367a2-a961-4f17-89b7-54cf99d1e5a3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:13.029939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62b52c-5375-4353-b477-1d768f8f86dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:13.030494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62b52c-5375-4353-b477-1d768f8f86dc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:13.030775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62b52c-5375-4353-b477-1d768f8f86dc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:13.031068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62b52c-5375-4353-b477-1d768f8f86dc None admin] PATH_INFO: `/v3/users/d25ddc64aaf24cd29aef2ebfe0019f67` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:13.032274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62b52c-5375-4353-b477-1d768f8f86dc None admin] RBAC: Authorizing `identity:get_user(user_id=d25ddc64aaf24cd29aef2ebfe0019f67)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:13.034018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62b52c-5375-4353-b477-1d768f8f86dc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:13.036080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 157/315] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:12 2022] GET /identity/v3/users/d25ddc64aaf24cd29aef2ebfe0019f67 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:13.042100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:13.160565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:13.165050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5d289-328c-453a-b3e3-e7244ecfec6b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:13.207997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397178e-8922-48d4-b55c-087348a096cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:13.209091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397178e-8922-48d4-b55c-087348a096cf None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:13.209625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397178e-8922-48d4-b55c-087348a096cf None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:13.210074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397178e-8922-48d4-b55c-087348a096cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:13.292452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397178e-8922-48d4-b55c-087348a096cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:13.293889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397178e-8922-48d4-b55c-087348a096cf None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:13.448107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:13.455729 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 159/316] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 13:16:13.465251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62b52c-5375-4353-b477-1d768f8f86dc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:13.661314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62b52c-5375-4353-b477-1d768f8f86dc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:13.709821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:13.725106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73574b-47b6-4be2-beb9-3d3611e191c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:13.726546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73574b-47b6-4be2-beb9-3d3611e191c2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:13.727275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73574b-47b6-4be2-beb9-3d3611e191c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:13.727953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73574b-47b6-4be2-beb9-3d3611e191c2 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:13.730241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73574b-47b6-4be2-beb9-3d3611e191c2 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:13.733227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73574b-47b6-4be2-beb9-3d3611e191c2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:13.737756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 158/317] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:13 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:13.748204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397178e-8922-48d4-b55c-087348a096cf None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:13.945367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397178e-8922-48d4-b55c-087348a096cf None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:14.014510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c51611-46db-4d4b-81a1-7f63281d0083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:14.015852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51611-46db-4d4b-81a1-7f63281d0083 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:14.016548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51611-46db-4d4b-81a1-7f63281d0083 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:14.017268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c51611-46db-4d4b-81a1-7f63281d0083 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:14.019017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51611-46db-4d4b-81a1-7f63281d0083 None admin] RBAC: Adding query filter params (role.id=cd3aa49828434199911a5124c59be2a4, scope.project.id=5a96496b71e840f8b6e17be47512394b, user.id=d25ddc64aaf24cd29aef2ebfe0019f67) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:14.019772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51611-46db-4d4b-81a1-7f63281d0083 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:14.022608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c51611-46db-4d4b-81a1-7f63281d0083 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:14.035047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 160/318] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:13 2022] GET /identity/v3/role_assignments?user.id=d25ddc64aaf24cd29aef2ebfe0019f67&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=cd3aa49828434199911a5124c59be2a4 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:14.488037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfcdc0-389b-46c3-93eb-3d69dd6d4c38 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:14.488604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfcdc0-389b-46c3-93eb-3d69dd6d4c38 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:14.488604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfcdc0-389b-46c3-93eb-3d69dd6d4c38 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:14.490110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 159/319] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:14.495557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c4fdb-5573-4a33-8400-0d9ed146b472 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:14.495868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c4fdb-5573-4a33-8400-0d9ed146b472 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:14.496128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c4fdb-5573-4a33-8400-0d9ed146b472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:14.515729 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-101c4fdb-5573-4a33-8400-0d9ed146b472 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:14.516961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-101c4fdb-5573-4a33-8400-0d9ed146b472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:14.581011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:14.590820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 161/320] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:14.865510 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfcdc0-389b-46c3-93eb-3d69dd6d4c38 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:15.049735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfcdc0-389b-46c3-93eb-3d69dd6d4c38 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:15.076287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e323d-e4a0-4233-a71e-6fdf3e0c7585 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:15.076836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e323d-e4a0-4233-a71e-6fdf3e0c7585 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:15.076836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81e323d-e4a0-4233-a71e-6fdf3e0c7585 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:15.078494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 162/321] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:15.084456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2893-e834-4ae8-9796-6b948ae0f056 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:15.084743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2893-e834-4ae8-9796-6b948ae0f056 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:15.085018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e2893-e834-4ae8-9796-6b948ae0f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:15.106838 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-789e2893-e834-4ae8-9796-6b948ae0f056 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:15.108635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aca6a-79af-4cfc-9235-cb35f366a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:15.108845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-789e2893-e834-4ae8-9796-6b948ae0f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:15.109283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aca6a-79af-4cfc-9235-cb35f366a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:15.109600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aca6a-79af-4cfc-9235-cb35f366a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:15.109878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aca6a-79af-4cfc-9235-cb35f366a6e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:15.111022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aca6a-79af-4cfc-9235-cb35f366a6e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:15.113774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503aca6a-79af-4cfc-9235-cb35f366a6e8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:15.115772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 160/322] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:15.121268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aca6a-79af-4cfc-9235-cb35f366a6e8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:15.199360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:15.231063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 163/323] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:15.249200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503aca6a-79af-4cfc-9235-cb35f366a6e8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:15.294129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de653e-6f78-4f9e-9fd6-d8deb4b636f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:15.295020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de653e-6f78-4f9e-9fd6-d8deb4b636f2 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:15.295456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de653e-6f78-4f9e-9fd6-d8deb4b636f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:15.295921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de653e-6f78-4f9e-9fd6-d8deb4b636f2 None admin] PATH_INFO: `/v3/groups` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:15.296742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de653e-6f78-4f9e-9fd6-d8deb4b636f2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:15.298467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de653e-6f78-4f9e-9fd6-d8deb4b636f2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:15.321563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 161/324] 10.222.0.58 () {60 vars in 1134 bytes} [Mon Oct 3 13:16:15 2022] POST /identity/v3/groups => generated 226 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:16:15.406165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bfb9e-4801-4ff2-a5bc-5d51f917da54 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:15.406586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bfb9e-4801-4ff2-a5bc-5d51f917da54 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:15.407132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bfb9e-4801-4ff2-a5bc-5d51f917da54 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:15.410626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 164/325] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:15.420058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed41a78-8987-4561-87af-65960c1380b3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:15.420557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed41a78-8987-4561-87af-65960c1380b3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:15.421059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed41a78-8987-4561-87af-65960c1380b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:15.452998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4ed41a78-8987-4561-87af-65960c1380b3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:15.455140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4ed41a78-8987-4561-87af-65960c1380b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:15.524286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bfb9e-4801-4ff2-a5bc-5d51f917da54 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:15.564458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 162/326] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:15.683903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bfb9e-4801-4ff2-a5bc-5d51f917da54 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:15.716744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:15.730094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de45fe-5c99-48fc-90d4-608d20d2c902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:15.730747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de45fe-5c99-48fc-90d4-608d20d2c902 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:15.731032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de45fe-5c99-48fc-90d4-608d20d2c902 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:15.731308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de45fe-5c99-48fc-90d4-608d20d2c902 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:15.744888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de45fe-5c99-48fc-90d4-608d20d2c902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-74de45fe-5c99-48fc-90d4-608d20d2c902 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:15.749984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:15.752955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:15.754668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 165/327] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:15.758232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed41a78-8987-4561-87af-65960c1380b3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:15.830350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:15.872041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de45fe-5c99-48fc-90d4-608d20d2c902 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:15.918857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed41a78-8987-4561-87af-65960c1380b3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:15.965504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:15.966014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:15.966295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:15.966570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:15.967258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:15.967564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:15.970859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:15.991763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 163/328] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:15.998554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:16.053327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de45fe-5c99-48fc-90d4-608d20d2c902 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:16.100789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee8730-f2df-480e-bc80-e12e94738d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:16.101613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee8730-f2df-480e-bc80-e12e94738d46 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:16.101773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee8730-f2df-480e-bc80-e12e94738d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:16.102156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee8730-f2df-480e-bc80-e12e94738d46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:16.121374 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee8730-f2df-480e-bc80-e12e94738d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-dcee8730-f2df-480e-bc80-e12e94738d46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:16.126339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:16.128346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:16.129412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 166/329] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:16.145107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee8730-f2df-480e-bc80-e12e94738d46 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:16.186572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9475c6a-1963-4df5-9c28-6ff40543f02d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:16.209745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:16.212762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:16.240091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd96e2-c01f-4f5e-bd7e-d55b226a54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:16.241882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd96e2-c01f-4f5e-bd7e-d55b226a54d5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:16.242357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd96e2-c01f-4f5e-bd7e-d55b226a54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:16.242780 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd96e2-c01f-4f5e-bd7e-d55b226a54d5 None admin] PATH_INFO: `/v3/users/10c1b3f9599043a68598e07801183dee` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:16.244202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd96e2-c01f-4f5e-bd7e-d55b226a54d5 None admin] RBAC: Authorizing `identity:get_user(user_id=10c1b3f9599043a68598e07801183dee)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:16.246966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd96e2-c01f-4f5e-bd7e-d55b226a54d5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:16.249537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 164/330] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:15 2022] GET /identity/v3/users/10c1b3f9599043a68598e07801183dee => generated 293 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:16.256937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd96e2-c01f-4f5e-bd7e-d55b226a54d5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:16.318232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee8730-f2df-480e-bc80-e12e94738d46 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:16.381228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:16.382522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:16.382955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:16.383414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:16.384243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:16.384710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:16.386425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:16.398663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 167/331] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:16.405121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:16.427383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd96e2-c01f-4f5e-bd7e-d55b226a54d5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:16.477212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:16.478055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:16.478439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:16.478817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:16.479679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] RBAC: Adding query filter params (role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, user.id=10c1b3f9599043a68598e07801183dee, scope.system=all) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:16.480107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:16.482855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:16.504346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 165/332] 10.222.0.58 () {56 vars in 1308 bytes} [Mon Oct 3 13:16:16 2022] GET /identity/v3/role_assignments?user.id=10c1b3f9599043a68598e07801183dee&scope.system=all&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 510 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:16.554250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31991794-1c4a-4680-91eb-bbdd1d4a262d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:16.581860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:16.600676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d4d11-118c-4910-9201-8e6344e4f127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:16.601353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4d11-118c-4910-9201-8e6344e4f127 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:16.601892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4d11-118c-4910-9201-8e6344e4f127 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:16.602271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4d11-118c-4910-9201-8e6344e4f127 None admin] PATH_INFO: `/v3/users/46c98e88c1bf474895eda8f950ca4f5c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:16.603596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d4d11-118c-4910-9201-8e6344e4f127 None admin] RBAC: Authorizing `identity:get_user(user_id=46c98e88c1bf474895eda8f950ca4f5c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:16.605106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d4d11-118c-4910-9201-8e6344e4f127 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:16.614396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 168/333] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:16 2022] GET /identity/v3/users/46c98e88c1bf474895eda8f950ca4f5c => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:16.615677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:16.718913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:16.794596 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682141e-6c10-486c-ae69-6b63297fcfd4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:16.835933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2be1e-4e1b-4a1a-865c-5df0817e7cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:16.836615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2be1e-4e1b-4a1a-865c-5df0817e7cb6 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:16.836897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2be1e-4e1b-4a1a-865c-5df0817e7cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:16.837173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2be1e-4e1b-4a1a-865c-5df0817e7cb6 None admin] PATH_INFO: `/v3/system/users/46c98e88c1bf474895eda8f950ca4f5c/roles/cd3aa49828434199911a5124c59be2a4` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:16.838689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2be1e-4e1b-4a1a-865c-5df0817e7cb6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=46c98e88c1bf474895eda8f950ca4f5c, role_id=cd3aa49828434199911a5124c59be2a4)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:16.840186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2be1e-4e1b-4a1a-865c-5df0817e7cb6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:16.850989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 166/334] 10.222.0.58 () {58 vars in 1334 bytes} [Mon Oct 3 13:16:16 2022] PUT /identity/v3/system/users/46c98e88c1bf474895eda8f950ca4f5c/roles/cd3aa49828434199911a5124c59be2a4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:16:18.380615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afaa1a-9f38-42b5-8507-be89123dabf9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:18.381138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afaa1a-9f38-42b5-8507-be89123dabf9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:18.381249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afaa1a-9f38-42b5-8507-be89123dabf9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:18.383049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 169/335] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:18.389138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428db8e2-6fb2-4792-8e1c-46bbda5ae1fe None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:18.390183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428db8e2-6fb2-4792-8e1c-46bbda5ae1fe None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:18.390694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428db8e2-6fb2-4792-8e1c-46bbda5ae1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:18.412620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-428db8e2-6fb2-4792-8e1c-46bbda5ae1fe None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:18.414272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-428db8e2-6fb2-4792-8e1c-46bbda5ae1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:18.514301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 167/336] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:18.957321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018c23-80f6-4ea2-b925-eba94aefe4d9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:18.957815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018c23-80f6-4ea2-b925-eba94aefe4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:18.958171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018c23-80f6-4ea2-b925-eba94aefe4d9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:18.960803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 170/337] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:18.970166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74f6f-8fe7-4ef5-8c96-2039e913aae7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:18.971044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74f6f-8fe7-4ef5-8c96-2039e913aae7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:18.972049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74f6f-8fe7-4ef5-8c96-2039e913aae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:19.027300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a8c74f6f-8fe7-4ef5-8c96-2039e913aae7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:19.029437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a8c74f6f-8fe7-4ef5-8c96-2039e913aae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:19.098690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:19.199967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 168/338] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:19.277180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91018c23-80f6-4ea2-b925-eba94aefe4d9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:19.523984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91018c23-80f6-4ea2-b925-eba94aefe4d9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:19.603189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a702c8c-3c0d-4e66-9913-3fd25714ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:19.604273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a702c8c-3c0d-4e66-9913-3fd25714ef4d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:19.604758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a702c8c-3c0d-4e66-9913-3fd25714ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:19.605249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a702c8c-3c0d-4e66-9913-3fd25714ef4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:19.627695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a702c8c-3c0d-4e66-9913-3fd25714ef4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3a702c8c-3c0d-4e66-9913-3fd25714ef4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:19.639678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:19.641435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:19.643589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 171/339] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:16:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:19.652235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74f6f-8fe7-4ef5-8c96-2039e913aae7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:19.750502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355bf95-b9a0-40e9-bd6b-9d6febe4f9ef None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:19.751522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355bf95-b9a0-40e9-bd6b-9d6febe4f9ef None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:19.752880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355bf95-b9a0-40e9-bd6b-9d6febe4f9ef None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:19.755842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 172/340] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:19.816019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93e441-d8a7-44ba-b60e-5b1865897188 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:19.817033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93e441-d8a7-44ba-b60e-5b1865897188 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:19.817820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93e441-d8a7-44ba-b60e-5b1865897188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:19.879146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74f6f-8fe7-4ef5-8c96-2039e913aae7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:19.879389 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8c93e441-d8a7-44ba-b60e-5b1865897188 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:19.881611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8c93e441-d8a7-44ba-b60e-5b1865897188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:19.962542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:19.963600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:19.964097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:19.964583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:19.965818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:19.966336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:19.968809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:19.980243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:20.005543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 169/341] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:20.025867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:20.117777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:20.126672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 173/342] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:20.208051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93e441-d8a7-44ba-b60e-5b1865897188 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:20.297310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec975dd-e6af-4617-9ba2-5e517c18b861 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:20.394980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4555e-6ee5-4a6e-9450-f04705fb1b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:20.396605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4555e-6ee5-4a6e-9450-f04705fb1b50 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:20.397388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4555e-6ee5-4a6e-9450-f04705fb1b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:20.398229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4555e-6ee5-4a6e-9450-f04705fb1b50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:20.481059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93e441-d8a7-44ba-b60e-5b1865897188 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:20.565785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4555e-6ee5-4a6e-9450-f04705fb1b50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:20.567347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4555e-6ee5-4a6e-9450-f04705fb1b50 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:20.572895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38e521-708c-4132-b7fe-95cdbb70fda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:20.573711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38e521-708c-4132-b7fe-95cdbb70fda5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:20.573839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38e521-708c-4132-b7fe-95cdbb70fda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:20.574164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38e521-708c-4132-b7fe-95cdbb70fda5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:20.601192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38e521-708c-4132-b7fe-95cdbb70fda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8d38e521-708c-4132-b7fe-95cdbb70fda5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:20.612084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:20.614190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:20.616125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 174/343] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:20.628475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38e521-708c-4132-b7fe-95cdbb70fda5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:20.682766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:20.688903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:20.823767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 170/344] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:20.835216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4555e-6ee5-4a6e-9450-f04705fb1b50 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:20.895827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38e521-708c-4132-b7fe-95cdbb70fda5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:20.967636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:20.969179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:20.969773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:20.970343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:20.972192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:20.972808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:20.976248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:20.993001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 175/345] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:20.999085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:21.085164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4555e-6ee5-4a6e-9450-f04705fb1b50 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:21.121946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:21.143751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1a5d6-bc61-457c-9d37-c7b5b2e35528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:21.144794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a5d6-bc61-457c-9d37-c7b5b2e35528 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.145272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a5d6-bc61-457c-9d37-c7b5b2e35528 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.146105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a5d6-bc61-457c-9d37-c7b5b2e35528 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.148410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1a5d6-bc61-457c-9d37-c7b5b2e35528 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:21.151017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1a5d6-bc61-457c-9d37-c7b5b2e35528 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:21.154591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 171/346] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:20 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:21.157180 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be1d6b-6462-456f-899c-d7ccb7ea1d52 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.161853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be1d6b-6462-456f-899c-d7ccb7ea1d52 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.162408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be1d6b-6462-456f-899c-d7ccb7ea1d52 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.164789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 172/347] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:21 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:21.166161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be1d6b-6462-456f-899c-d7ccb7ea1d52 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:21.193134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:21.224766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51e6ad-2bb0-4327-a0f4-55b5770667b6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:21.272660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d154b61-0cf4-4f94-9664-9fd775ff0f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:21.273330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d154b61-0cf4-4f94-9664-9fd775ff0f49 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.273654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d154b61-0cf4-4f94-9664-9fd775ff0f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.273935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d154b61-0cf4-4f94-9664-9fd775ff0f49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.295431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d154b61-0cf4-4f94-9664-9fd775ff0f49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2d154b61-0cf4-4f94-9664-9fd775ff0f49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:21.303775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:21.306188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 176/348] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:21.308066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9311d-8370-47c8-8335-f05a76dc9834 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.310178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9311d-8370-47c8-8335-f05a76dc9834 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.310178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9311d-8370-47c8-8335-f05a76dc9834 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.310178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 177/349] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:21.310178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bb147-fde6-4e10-a9c7-83a10da56dcd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.310697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bb147-fde6-4e10-a9c7-83a10da56dcd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.310697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bb147-fde6-4e10-a9c7-83a10da56dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.342829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-121bb147-fde6-4e10-a9c7-83a10da56dcd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:21.342829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-121bb147-fde6-4e10-a9c7-83a10da56dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:21.428121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be1d6b-6462-456f-899c-d7ccb7ea1d52 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:21.451680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 178/350] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:21.452554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bb147-fde6-4e10-a9c7-83a10da56dcd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:21.482330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44042919-483d-47da-9d6f-1afcbd0d17e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:21.482330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44042919-483d-47da-9d6f-1afcbd0d17e4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.482718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44042919-483d-47da-9d6f-1afcbd0d17e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.482848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44042919-483d-47da-9d6f-1afcbd0d17e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.590276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:21.605942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:21.614303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44042919-483d-47da-9d6f-1afcbd0d17e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:21.615912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44042919-483d-47da-9d6f-1afcbd0d17e4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:21.649387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bb147-fde6-4e10-a9c7-83a10da56dcd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:21.704021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b071fd-79f3-4dc4-b076-5b0b70e5dccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:21.706193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b071fd-79f3-4dc4-b076-5b0b70e5dccf None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.706495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b071fd-79f3-4dc4-b076-5b0b70e5dccf None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.706776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b071fd-79f3-4dc4-b076-5b0b70e5dccf None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.716028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b071fd-79f3-4dc4-b076-5b0b70e5dccf None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:21.717833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b071fd-79f3-4dc4-b076-5b0b70e5dccf None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:21.720059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 179/351] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:21 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:21.721363 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb341-8584-4752-ac05-e2611715717a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.721892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb341-8584-4752-ac05-e2611715717a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.722177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffb341-8584-4752-ac05-e2611715717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.749920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-31ffb341-8584-4752-ac05-e2611715717a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:21.751197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-31ffb341-8584-4752-ac05-e2611715717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:21.783637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 173/352] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:21.783637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44042919-483d-47da-9d6f-1afcbd0d17e4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:21.858309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 180/353] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:21.859685 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb1083-19b1-4ef8-ae63-52755804295f None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:21.860058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb1083-19b1-4ef8-ae63-52755804295f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:21.860347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb1083-19b1-4ef8-ae63-52755804295f None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:21.864288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 181/354] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:21.864288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb1083-19b1-4ef8-ae63-52755804295f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:21.930809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:21.941339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44042919-483d-47da-9d6f-1afcbd0d17e4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:22.005680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:22.007108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:22.007692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:22.008231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:22.009774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:22.010294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:22.010400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:22.015039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:22.035371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 174/355] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:22.037162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:22.076453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb1083-19b1-4ef8-ae63-52755804295f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:22.149562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdefd5-e424-4217-b6cf-c7ddb944b035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:22.150491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdefd5-e424-4217-b6cf-c7ddb944b035 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:22.150776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdefd5-e424-4217-b6cf-c7ddb944b035 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:22.151051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdefd5-e424-4217-b6cf-c7ddb944b035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:22.151807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdefd5-e424-4217-b6cf-c7ddb944b035 None admin] RBAC: Adding query filter params (group.id=f4037f7fa305455fb12f132d2f0c36af, role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=dda4727fec9747b4b20cb990f7305e3d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:22.152119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdefd5-e424-4217-b6cf-c7ddb944b035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:22.153646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdefd5-e424-4217-b6cf-c7ddb944b035 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:22.160785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 182/356] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:21 2022] GET /identity/v3/role_assignments?group.id=f4037f7fa305455fb12f132d2f0c36af&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 260 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:22.162443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0a20a-11f3-4ad5-b4cd-e02117e2581a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:22.162992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0a20a-11f3-4ad5-b4cd-e02117e2581a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:22.163474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0a20a-11f3-4ad5-b4cd-e02117e2581a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:22.185216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a3c0a20a-11f3-4ad5-b4cd-e02117e2581a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:22.186647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a3c0a20a-11f3-4ad5-b4cd-e02117e2581a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:22.223616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82806a3a-9d3e-4030-b8e4-3c2ca91d6ce6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:22.284302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db46f25-0877-4dc0-ac29-04e7b49c7c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:22.285411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db46f25-0877-4dc0-ac29-04e7b49c7c33 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:22.286191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db46f25-0877-4dc0-ac29-04e7b49c7c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:22.286684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db46f25-0877-4dc0-ac29-04e7b49c7c33 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:22.288725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db46f25-0877-4dc0-ac29-04e7b49c7c33 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:22.290592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db46f25-0877-4dc0-ac29-04e7b49c7c33 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:22.298172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 175/357] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:22 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:22.298606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db46f25-0877-4dc0-ac29-04e7b49c7c33 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:22.326527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 183/358] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:22.327506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0a20a-11f3-4ad5-b4cd-e02117e2581a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:22.366574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:22.467087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db46f25-0877-4dc0-ac29-04e7b49c7c33 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:22.474906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:22.522707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52348c1a-6f0b-4cf7-8733-d881634167f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:22.523983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348c1a-6f0b-4cf7-8733-d881634167f3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:22.524605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348c1a-6f0b-4cf7-8733-d881634167f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:22.525222 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52348c1a-6f0b-4cf7-8733-d881634167f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:22.545994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0a20a-11f3-4ad5-b4cd-e02117e2581a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:22.554127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52348c1a-6f0b-4cf7-8733-d881634167f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-52348c1a-6f0b-4cf7-8733-d881634167f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:22.564305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:22.565391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:22.566254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:22.573547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 176/359] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:16:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:22.574884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52348c1a-6f0b-4cf7-8733-d881634167f3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:22.630923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33f71-24ea-4f5f-87b5-56ad3c01a7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:22.632642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33f71-24ea-4f5f-87b5-56ad3c01a7ba None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:22.633502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33f71-24ea-4f5f-87b5-56ad3c01a7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:22.634280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33f71-24ea-4f5f-87b5-56ad3c01a7ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:22.738678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:22.767858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b33f71-24ea-4f5f-87b5-56ad3c01a7ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:22.771142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b33f71-24ea-4f5f-87b5-56ad3c01a7ba None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:22.781152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52348c1a-6f0b-4cf7-8733-d881634167f3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:22.863777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:22.868614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e707f6-2ef8-4f3e-b54c-cecd4152e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:22.870705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e707f6-2ef8-4f3e-b54c-cecd4152e920 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:22.871883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e707f6-2ef8-4f3e-b54c-cecd4152e920 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:22.874601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e707f6-2ef8-4f3e-b54c-cecd4152e920 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:22.905272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e707f6-2ef8-4f3e-b54c-cecd4152e920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-70e707f6-2ef8-4f3e-b54c-cecd4152e920 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:22.917089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:22.918151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 03 13:16:22.918965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:22.920166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 177/360] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:22.921436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e707f6-2ef8-4f3e-b54c-cecd4152e920 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:22.927203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 184/361] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:22.928925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33f71-24ea-4f5f-87b5-56ad3c01a7ba None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:23.091005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e707f6-2ef8-4f3e-b54c-cecd4152e920 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:23.124835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33f71-24ea-4f5f-87b5-56ad3c01a7ba None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:23.150190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdbe58-fdf5-47ca-87aa-65cf932fc4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:23.150832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdbe58-fdf5-47ca-87aa-65cf932fc4fe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:23.151293 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdbe58-fdf5-47ca-87aa-65cf932fc4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:23.151745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdbe58-fdf5-47ca-87aa-65cf932fc4fe None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:23.153885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdbe58-fdf5-47ca-87aa-65cf932fc4fe None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:23.156500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdbe58-fdf5-47ca-87aa-65cf932fc4fe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:23.160470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 178/362] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:22 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:23.167767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdbe58-fdf5-47ca-87aa-65cf932fc4fe None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:23.198356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:23.199452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:23.199854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:23.200320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:23.200772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:23.202306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:23.202821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:23.206960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:23.225548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 185/363] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:23.231108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:23.292650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:23.395511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdbe58-fdf5-47ca-87aa-65cf932fc4fe None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:23.441256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdbadb-2b8b-4bff-81ab-f28384974099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:23.441949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdbadb-2b8b-4bff-81ab-f28384974099 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:23.442237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdbadb-2b8b-4bff-81ab-f28384974099 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:23.442520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdbadb-2b8b-4bff-81ab-f28384974099 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:23.446359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff608d-1754-4505-93ee-47ad86b5d6e5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:23.454755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdbadb-2b8b-4bff-81ab-f28384974099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6fbdbadb-2b8b-4bff-81ab-f28384974099 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:23.460430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:23.462331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 179/364] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:23.468171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdbadb-2b8b-4bff-81ab-f28384974099 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:23.507469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:23.508396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:23.508680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:23.508968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:23.509699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:23.510055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:23.511431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:23.529658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 186/365] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:23.531541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:23.583016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:23.625530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdbadb-2b8b-4bff-81ab-f28384974099 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:23.673122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:23.674379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e49d8-b421-4642-b8d9-b24618d2ff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:23.674832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e49d8-b421-4642-b8d9-b24618d2ff08 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:23.675354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e49d8-b421-4642-b8d9-b24618d2ff08 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:23.675830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e49d8-b421-4642-b8d9-b24618d2ff08 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:23.677871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e49d8-b421-4642-b8d9-b24618d2ff08 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:23.680259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e49d8-b421-4642-b8d9-b24618d2ff08 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:23.683496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 180/366] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:23 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:23.684280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e49d8-b421-4642-b8d9-b24618d2ff08 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:23.698765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c8c8-4698-4672-a9ce-287f1ee54c21 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:23.765085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:23.766327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:23.766731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:23.768157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:23.769195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] RBAC: Adding query filter params (group.id=f4037f7fa305455fb12f132d2f0c36af, role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=dda4727fec9747b4b20cb990f7305e3d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:23.769802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:23.772395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:23.793500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 187/367] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:23 2022] GET /identity/v3/role_assignments?group.id=f4037f7fa305455fb12f132d2f0c36af&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 260 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:23.793500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:23.849081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e49d8-b421-4642-b8d9-b24618d2ff08 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:23.934804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7bb91-3fa7-44e1-a826-8f04ead345ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:23.937014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7bb91-3fa7-44e1-a826-8f04ead345ae None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:23.937014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7bb91-3fa7-44e1-a826-8f04ead345ae None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:23.937014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7bb91-3fa7-44e1-a826-8f04ead345ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:23.960986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:23.997432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cb978-0571-43c2-8ad7-3a0465009c66 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:24.053223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:24.053933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:24.054218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:24.054816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:24.055504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:24.055815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:24.058014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:24.060837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7bb91-3fa7-44e1-a826-8f04ead345ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:24.063817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7bb91-3fa7-44e1-a826-8f04ead345ae None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:24.068720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 188/368] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:24.070895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:24.099367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:24.225096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 181/369] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:24.226488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7bb91-3fa7-44e1-a826-8f04ead345ae None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:24.249612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c42f85-2066-4699-8880-0007a9217bcb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:24.296823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b347565-8888-44a4-9548-ecc636a4d64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:24.299281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b347565-8888-44a4-9548-ecc636a4d64c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:24.299833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b347565-8888-44a4-9548-ecc636a4d64c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:24.300308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b347565-8888-44a4-9548-ecc636a4d64c None admin] PATH_INFO: `/v3/users/46c98e88c1bf474895eda8f950ca4f5c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:24.304164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b347565-8888-44a4-9548-ecc636a4d64c None admin] RBAC: Authorizing `identity:get_user(user_id=46c98e88c1bf474895eda8f950ca4f5c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:24.306714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b347565-8888-44a4-9548-ecc636a4d64c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:24.307783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:24.310587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 189/370] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:16:24 2022] GET /identity/v3/users/46c98e88c1bf474895eda8f950ca4f5c => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:24.315897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b347565-8888-44a4-9548-ecc636a4d64c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:24.418679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7bb91-3fa7-44e1-a826-8f04ead345ae None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:24.432873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:24.467196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b347565-8888-44a4-9548-ecc636a4d64c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:24.473310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086f3b1-1d34-4d4b-b6fc-7567ba74e984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:24.474349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f3b1-1d34-4d4b-b6fc-7567ba74e984 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:24.474871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f3b1-1d34-4d4b-b6fc-7567ba74e984 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:24.475403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f3b1-1d34-4d4b-b6fc-7567ba74e984 None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:24.477374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086f3b1-1d34-4d4b-b6fc-7567ba74e984 None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:24.479291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086f3b1-1d34-4d4b-b6fc-7567ba74e984 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:24.486908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 182/371] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:24 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:24.486908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086f3b1-1d34-4d4b-b6fc-7567ba74e984 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:24.525880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f2105-b53f-43bc-a3e5-ce432584b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:24.527379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f2105-b53f-43bc-a3e5-ce432584b531 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:24.528073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f2105-b53f-43bc-a3e5-ce432584b531 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:24.528496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f2105-b53f-43bc-a3e5-ce432584b531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:24.612868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086f3b1-1d34-4d4b-b6fc-7567ba74e984 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:24.624969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:24.632399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f2105-b53f-43bc-a3e5-ce432584b531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:24.634445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f2105-b53f-43bc-a3e5-ce432584b531 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:24.663599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024023-e9c3-46d9-b4e3-1f62062c7e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:24.664874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024023-e9c3-46d9-b4e3-1f62062c7e0e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:24.665305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024023-e9c3-46d9-b4e3-1f62062c7e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:24.666223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c024023-e9c3-46d9-b4e3-1f62062c7e0e None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:24.667607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024023-e9c3-46d9-b4e3-1f62062c7e0e None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:24.669286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c024023-e9c3-46d9-b4e3-1f62062c7e0e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:24.671217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 183/372] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:24 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:24.672177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024023-e9c3-46d9-b4e3-1f62062c7e0e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:24.791154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:24.811885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c024023-e9c3-46d9-b4e3-1f62062c7e0e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:24.836230 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 190/373] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:24.838062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f2105-b53f-43bc-a3e5-ce432584b531 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:24.870363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:24.871443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:24.871981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:24.872415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:24.873989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] RBAC: Adding query filter params (role.id=cd3aa49828434199911a5124c59be2a4, user.id=46c98e88c1bf474895eda8f950ca4f5c, scope.system=all) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:24.874534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:24.877433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:24.909479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 184/374] 10.222.0.58 () {56 vars in 1308 bytes} [Mon Oct 3 13:16:24 2022] GET /identity/v3/role_assignments?user.id=46c98e88c1bf474895eda8f950ca4f5c&scope.system=all&role.id=cd3aa49828434199911a5124c59be2a4 => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:24.909479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:24.987335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:25.029402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f2105-b53f-43bc-a3e5-ce432584b531 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:25.084685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57e1f9-aa79-4303-852c-d976b9becfe2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:25.084900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:25.085497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:25.085812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:25.086179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:25.086925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] RBAC: Adding query filter params (group.id=f4037f7fa305455fb12f132d2f0c36af, role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=5a96496b71e840f8b6e17be47512394b) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:25.087320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:25.088962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:25.096126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 191/375] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:24 2022] GET /identity/v3/role_assignments?group.id=f4037f7fa305455fb12f132d2f0c36af&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:25.096816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:25.168078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:25.178372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f0148-4660-4825-ab6c-56ca4a71dee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:25.179043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0148-4660-4825-ab6c-56ca4a71dee4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:25.179630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0148-4660-4825-ab6c-56ca4a71dee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:25.179916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f0148-4660-4825-ab6c-56ca4a71dee4 None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:25.183710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f0148-4660-4825-ab6c-56ca4a71dee4 None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:25.186669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f0148-4660-4825-ab6c-56ca4a71dee4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:25.189586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 185/376] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:24 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:25.191709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2141a-0f38-410a-8fdd-ddf89a02faf8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:25.192279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2141a-0f38-410a-8fdd-ddf89a02faf8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:25.192563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2141a-0f38-410a-8fdd-ddf89a02faf8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:25.194133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 186/377] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:25.198760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2141a-0f38-410a-8fdd-ddf89a02faf8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:25.292473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97230d26-5fd7-4c5a-88bc-219db59d481b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:25.368696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2141a-0f38-410a-8fdd-ddf89a02faf8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:25.372564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:25.394944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de7233-e36c-4cde-bce6-a5b506439788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:25.395619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7233-e36c-4cde-bce6-a5b506439788 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:25.395907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7233-e36c-4cde-bce6-a5b506439788 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:25.396190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de7233-e36c-4cde-bce6-a5b506439788 None admin] PATH_INFO: `/v3/projects/cf1b6fdf131f4a598a5705727a85c56c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:25.401834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de7233-e36c-4cde-bce6-a5b506439788 None admin] RBAC: Authorizing `identity:get_project(project_id=cf1b6fdf131f4a598a5705727a85c56c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:25.403337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de7233-e36c-4cde-bce6-a5b506439788 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:25.405404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 192/378] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:25 2022] GET /identity/v3/projects/cf1b6fdf131f4a598a5705727a85c56c => generated 344 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:25.406661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da73d9-2c37-4e53-b98d-70a10a8a06f5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:25.406978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da73d9-2c37-4e53-b98d-70a10a8a06f5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:25.407292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da73d9-2c37-4e53-b98d-70a10a8a06f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:25.415416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:25.416049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:25.416338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:25.416624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:25.417376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] RBAC: Adding query filter params (group.id=f4037f7fa305455fb12f132d2f0c36af, role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=5a96496b71e840f8b6e17be47512394b) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:25.417746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:25.419229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:25.436071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 187/379] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:25 2022] GET /identity/v3/role_assignments?group.id=f4037f7fa305455fb12f132d2f0c36af&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:25.439385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:25.446605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a1da73d9-2c37-4e53-b98d-70a10a8a06f5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:25.447884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a1da73d9-2c37-4e53-b98d-70a10a8a06f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:25.565840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 193/380] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:25.613686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3b993-1898-49f2-b91e-49a90352e02c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:25.702550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686ef7f-6546-477f-903d-ef06b1237e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:25.703346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686ef7f-6546-477f-903d-ef06b1237e8c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:25.703630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686ef7f-6546-477f-903d-ef06b1237e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:25.703908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686ef7f-6546-477f-903d-ef06b1237e8c None admin] PATH_INFO: `/v3/groups/2d344a51c948420586569fb7b54b400f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:25.716356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686ef7f-6546-477f-903d-ef06b1237e8c None admin] RBAC: Authorizing `identity:get_group(group_id=2d344a51c948420586569fb7b54b400f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:25.717905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9686ef7f-6546-477f-903d-ef06b1237e8c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:25.719612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 188/381] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:25 2022] GET /identity/v3/groups/2d344a51c948420586569fb7b54b400f => generated 229 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:25.726264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da73d9-2c37-4e53-b98d-70a10a8a06f5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:25.793980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:25.875181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da73d9-2c37-4e53-b98d-70a10a8a06f5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:25.915662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:25.916299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:25.916584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:25.916855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:25.917638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] RBAC: Adding query filter params (group.id=2d344a51c948420586569fb7b54b400f, role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=cf1b6fdf131f4a598a5705727a85c56c) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:25.917947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:25.919428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:25.922514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686ef7f-6546-477f-903d-ef06b1237e8c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:25.926938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 194/382] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:25 2022] GET /identity/v3/role_assignments?group.id=2d344a51c948420586569fb7b54b400f&scope.project.id=cf1b6fdf131f4a598a5705727a85c56c&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:25.968355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:26.075969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9686ef7f-6546-477f-903d-ef06b1237e8c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:26.144443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c33066-f130-48b8-8146-8e528d9e4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:26.146155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33066-f130-48b8-8146-8e528d9e4f9b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:26.147090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33066-f130-48b8-8146-8e528d9e4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:26.147875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c33066-f130-48b8-8146-8e528d9e4f9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:26.172761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c33066-f130-48b8-8146-8e528d9e4f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d9c33066-f130-48b8-8146-8e528d9e4f9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:26.184686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:26.186746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:26.188255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:26.194516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 189/383] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:16:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:26.202802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:26.434066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c5a05-22c5-44b4-b0aa-07d7bb7da90b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:26.455617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:26.507162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:26.507890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:26.508185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:26.508462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:26.509149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:26.509475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:26.511169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:26.512593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db7b74-7ab5-4b61-87cb-c185b8c21a63 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:26.512877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db7b74-7ab5-4b61-87cb-c185b8c21a63 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:26.513153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db7b74-7ab5-4b61-87cb-c185b8c21a63 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:26.514707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 190/384] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:26.521945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccfcc2-c3f9-4a1f-9d47-a8cff5dfc79c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:26.522265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccfcc2-c3f9-4a1f-9d47-a8cff5dfc79c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:26.522516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccfcc2-c3f9-4a1f-9d47-a8cff5dfc79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:26.543421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 195/385] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:26.551005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:26.564749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e3ccfcc2-c3f9-4a1f-9d47-a8cff5dfc79c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:26.566245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e3ccfcc2-c3f9-4a1f-9d47-a8cff5dfc79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:26.705761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 191/386] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:26.746352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73687f45-3daa-4b50-b8e7-c44f0da8408e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:26.787901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1efee-1bd1-4dac-8be7-c3966495a6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:26.788990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1efee-1bd1-4dac-8be7-c3966495a6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:26.789485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1efee-1bd1-4dac-8be7-c3966495a6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:26.789983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c1efee-1bd1-4dac-8be7-c3966495a6c1 None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:26.791359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1efee-1bd1-4dac-8be7-c3966495a6c1 None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:26.793016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c1efee-1bd1-4dac-8be7-c3966495a6c1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:26.794932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 196/387] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:26 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:26.804779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccfcc2-c3f9-4a1f-9d47-a8cff5dfc79c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:26.858256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:27.014855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccfcc2-c3f9-4a1f-9d47-a8cff5dfc79c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:27.084685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145a5d-6857-45b2-ad0f-3627e775e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:27.085941 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145a5d-6857-45b2-ad0f-3627e775e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:27.086350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145a5d-6857-45b2-ad0f-3627e775e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:27.086843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65145a5d-6857-45b2-ad0f-3627e775e3f2 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:27.089165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145a5d-6857-45b2-ad0f-3627e775e3f2 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:27.089549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1efee-1bd1-4dac-8be7-c3966495a6c1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:27.092176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65145a5d-6857-45b2-ad0f-3627e775e3f2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:27.101540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 192/388] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:26 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:27.110220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145a5d-6857-45b2-ad0f-3627e775e3f2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:27.238074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:27.405709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:27.439785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65145a5d-6857-45b2-ad0f-3627e775e3f2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:27.447840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c1efee-1bd1-4dac-8be7-c3966495a6c1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:27.514951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13323e-cb8d-400e-b059-1e44605a8f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:27.516151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13323e-cb8d-400e-b059-1e44605a8f54 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:27.516663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13323e-cb8d-400e-b059-1e44605a8f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:27.517747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13323e-cb8d-400e-b059-1e44605a8f54 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d/groups/f4037f7fa305455fb12f132d2f0c36af/roles/753d3bbfcdaa4cef86cc8de37ae19906` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:27.525494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13323e-cb8d-400e-b059-1e44605a8f54 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda4727fec9747b4b20cb990f7305e3d, group_id=f4037f7fa305455fb12f132d2f0c36af, role_id=753d3bbfcdaa4cef86cc8de37ae19906)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:27.531408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13323e-cb8d-400e-b059-1e44605a8f54 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:27.552020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc80bc1-d753-49ea-a0d8-0157a0a16f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:27.553159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc80bc1-d753-49ea-a0d8-0157a0a16f0f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:27.556800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc80bc1-d753-49ea-a0d8-0157a0a16f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:27.558566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc80bc1-d753-49ea-a0d8-0157a0a16f0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:27.570052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 193/389] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Oct 3 13:16:27 2022] PUT /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d/groups/f4037f7fa305455fb12f132d2f0c36af/roles/753d3bbfcdaa4cef86cc8de37ae19906 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:16:27.592352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc80bc1-d753-49ea-a0d8-0157a0a16f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-fbc80bc1-d753-49ea-a0d8-0157a0a16f0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:27.597960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:27.599698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:27.601251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 197/390] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 511 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:27.610966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13323e-cb8d-400e-b059-1e44605a8f54 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:27.827697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13323e-cb8d-400e-b059-1e44605a8f54 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:27.896489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:27.897478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:27.898070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:27.898565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:27.899824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:27.900376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:27.903160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:27.921421 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 194/391] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:27.927509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc80bc1-d753-49ea-a0d8-0157a0a16f0f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:28.097732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:28.102359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc80bc1-d753-49ea-a0d8-0157a0a16f0f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:28.158999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92308213-9de1-4bb9-9bb9-b2b50203693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:28.159585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92308213-9de1-4bb9-9bb9-b2b50203693a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:28.159875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92308213-9de1-4bb9-9bb9-b2b50203693a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:28.160157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92308213-9de1-4bb9-9bb9-b2b50203693a None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:28.161489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92308213-9de1-4bb9-9bb9-b2b50203693a None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:28.162974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92308213-9de1-4bb9-9bb9-b2b50203693a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:28.164965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 198/392] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:27 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:28.171168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:28.212585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:28.408714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a384494-6f97-4400-8a53-660ef6f87ea1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:28.450536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e356b76-d519-422b-ada7-d9366f03642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:28.451593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e356b76-d519-422b-ada7-d9366f03642e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:28.452081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e356b76-d519-422b-ada7-d9366f03642e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:28.452551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e356b76-d519-422b-ada7-d9366f03642e None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:28.453936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e356b76-d519-422b-ada7-d9366f03642e None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:28.455747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e356b76-d519-422b-ada7-d9366f03642e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:28.461826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 195/393] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:28 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:28.472888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92308213-9de1-4bb9-9bb9-b2b50203693a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:28.504815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05d6a-676a-4a25-87cd-da7440f57561 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:28.505437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05d6a-676a-4a25-87cd-da7440f57561 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:28.505937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05d6a-676a-4a25-87cd-da7440f57561 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:28.508473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 196/394] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:28.515372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bf922-79a5-4b9e-bcba-6963fc0d1ff4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:28.515981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bf922-79a5-4b9e-bcba-6963fc0d1ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:28.516654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201bf922-79a5-4b9e-bcba-6963fc0d1ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:28.542378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-201bf922-79a5-4b9e-bcba-6963fc0d1ff4 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:28.544013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-201bf922-79a5-4b9e-bcba-6963fc0d1ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:28.665188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 197/395] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:28.684810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92308213-9de1-4bb9-9bb9-b2b50203693a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:28.741044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee2505-7055-4852-954c-535dd38d42c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:28.741692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee2505-7055-4852-954c-535dd38d42c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:28.741982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee2505-7055-4852-954c-535dd38d42c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:28.742264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee2505-7055-4852-954c-535dd38d42c6 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d/groups/f4037f7fa305455fb12f132d2f0c36af/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:28.745676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee2505-7055-4852-954c-535dd38d42c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dda4727fec9747b4b20cb990f7305e3d, group_id=f4037f7fa305455fb12f132d2f0c36af, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:28.747188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee2505-7055-4852-954c-535dd38d42c6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:28.767242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 199/396] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Oct 3 13:16:28 2022] PUT /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d/groups/f4037f7fa305455fb12f132d2f0c36af/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:16:29.107613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bf922-79a5-4b9e-bcba-6963fc0d1ff4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:29.121970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:29.230632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201bf922-79a5-4b9e-bcba-6963fc0d1ff4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:29.231722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14d97-17c5-4064-b466-a608c89f2510 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:29.232239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14d97-17c5-4064-b466-a608c89f2510 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:29.232740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14d97-17c5-4064-b466-a608c89f2510 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:29.241633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 200/397] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:29.249479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf324e-f010-4e53-b179-85a4ceaf17b7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:29.249479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf324e-f010-4e53-b179-85a4ceaf17b7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:29.249479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf324e-f010-4e53-b179-85a4ceaf17b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:29.282693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60a441-bb38-4033-9b62-1e38599ed457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:29.283723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a441-bb38-4033-9b62-1e38599ed457 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:29.284255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a441-bb38-4033-9b62-1e38599ed457 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:29.284722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60a441-bb38-4033-9b62-1e38599ed457 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:29.296766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60a441-bb38-4033-9b62-1e38599ed457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:29.300603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9caf324e-f010-4e53-b179-85a4ceaf17b7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:29.303150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9caf324e-f010-4e53-b179-85a4ceaf17b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ff60a441-bb38-4033-9b62-1e38599ed457 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:29.303507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:29.304368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:29.305182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:29.307869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 198/398] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:29.313608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12317bf-fca0-4887-9a44-7f36f88d1536 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:29.314979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12317bf-fca0-4887-9a44-7f36f88d1536 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:29.315556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12317bf-fca0-4887-9a44-7f36f88d1536 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:29.317204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 199/399] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:29.319212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12317bf-fca0-4887-9a44-7f36f88d1536 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:29.365400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:29.419524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:29.463012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12317bf-fca0-4887-9a44-7f36f88d1536 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:29.482091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 201/400] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:29.483464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e93db-b5cc-4023-a732-6c2cfa75b04e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:29.483757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e93db-b5cc-4023-a732-6c2cfa75b04e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:29.484035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e93db-b5cc-4023-a732-6c2cfa75b04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:29.523097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-363e93db-b5cc-4023-a732-6c2cfa75b04e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:29.524471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-363e93db-b5cc-4023-a732-6c2cfa75b04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:29.528584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:29.529216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:29.529529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:29.529815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:29.530696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:29.531014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:29.532536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:29.550628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 200/401] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:29.556508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:29.728907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 202/402] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:29.872696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b014-38a5-4c13-8030-13418f113320 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:29.950089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:29.958028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa9e4a-11da-4c5f-8d1d-60fff42e41fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:29.959074 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9e4a-11da-4c5f-8d1d-60fff42e41fd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:29.959493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9e4a-11da-4c5f-8d1d-60fff42e41fd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:29.959847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa9e4a-11da-4c5f-8d1d-60fff42e41fd None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:29.962247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa9e4a-11da-4c5f-8d1d-60fff42e41fd None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:29.963805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa9e4a-11da-4c5f-8d1d-60fff42e41fd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:29.965484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 201/403] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:29 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:29.972627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e93db-b5cc-4023-a732-6c2cfa75b04e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:30.025401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa9e4a-11da-4c5f-8d1d-60fff42e41fd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:30.191732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:30.228681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e93db-b5cc-4023-a732-6c2cfa75b04e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:30.251410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa9e4a-11da-4c5f-8d1d-60fff42e41fd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:30.297760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250daab2-6fde-4002-b551-24acae306d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:30.299187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250daab2-6fde-4002-b551-24acae306d9c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:30.299612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250daab2-6fde-4002-b551-24acae306d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:30.300034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250daab2-6fde-4002-b551-24acae306d9c None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:30.303086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250daab2-6fde-4002-b551-24acae306d9c None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:30.306498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250daab2-6fde-4002-b551-24acae306d9c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:30.312128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 203/404] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:29 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:30.315587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250daab2-6fde-4002-b551-24acae306d9c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:30.340998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84215ec-4da6-40d5-8860-1f13576ba1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:30.342062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84215ec-4da6-40d5-8860-1f13576ba1ca None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:30.342498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84215ec-4da6-40d5-8860-1f13576ba1ca None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:30.342927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84215ec-4da6-40d5-8860-1f13576ba1ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:30.372031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84215ec-4da6-40d5-8860-1f13576ba1ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e84215ec-4da6-40d5-8860-1f13576ba1ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:30.383685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:30.385690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:30.387593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 202/405] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:16:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:30.389300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84215ec-4da6-40d5-8860-1f13576ba1ca None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:30.512761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:30.523504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250daab2-6fde-4002-b551-24acae306d9c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:30.591142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd0d4d-8f38-4a11-832a-3f04d6ed5295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:30.592217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0d4d-8f38-4a11-832a-3f04d6ed5295 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:30.592828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0d4d-8f38-4a11-832a-3f04d6ed5295 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:30.593188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0d4d-8f38-4a11-832a-3f04d6ed5295 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:30.607155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84215ec-4da6-40d5-8860-1f13576ba1ca None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:30.614097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd0d4d-8f38-4a11-832a-3f04d6ed5295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5fcd0d4d-8f38-4a11-832a-3f04d6ed5295 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:30.624215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:30.626247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 204/406] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:16:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:30.629033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd0d4d-8f38-4a11-832a-3f04d6ed5295 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:30.660974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:30.692825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eee77-ec7d-436c-b3a7-b0bc8995374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:30.694280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eee77-ec7d-436c-b3a7-b0bc8995374d None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:30.695008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eee77-ec7d-436c-b3a7-b0bc8995374d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:30.695715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eee77-ec7d-436c-b3a7-b0bc8995374d None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b/groups/f4037f7fa305455fb12f132d2f0c36af/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:30.699638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3eee77-ec7d-436c-b3a7-b0bc8995374d None admin] RBAC: Authorizing `identity:create_grant(project_id=5a96496b71e840f8b6e17be47512394b, group_id=f4037f7fa305455fb12f132d2f0c36af, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:30.702668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3eee77-ec7d-436c-b3a7-b0bc8995374d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:30.729329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 203/407] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Oct 3 13:16:30 2022] PUT /identity/v3/projects/5a96496b71e840f8b6e17be47512394b/groups/f4037f7fa305455fb12f132d2f0c36af/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:16:30.731349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eee77-ec7d-436c-b3a7-b0bc8995374d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:30.852617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd0d4d-8f38-4a11-832a-3f04d6ed5295 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:30.932947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9dc09-cf15-482d-b63e-b7fa5f6d9515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:30.934493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9dc09-cf15-482d-b63e-b7fa5f6d9515 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:30.935007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9dc09-cf15-482d-b63e-b7fa5f6d9515 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:30.937487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9dc09-cf15-482d-b63e-b7fa5f6d9515 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:30.938927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9dc09-cf15-482d-b63e-b7fa5f6d9515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:30.940150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9dc09-cf15-482d-b63e-b7fa5f6d9515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:30.943305 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9dc09-cf15-482d-b63e-b7fa5f6d9515 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:30.964265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 205/408] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:30.967531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eff23-cf06-4209-92c2-81df9149c0df None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:30.967531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eff23-cf06-4209-92c2-81df9149c0df None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:30.967531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eff23-cf06-4209-92c2-81df9149c0df None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:30.975597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 206/409] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:30.975597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eff23-cf06-4209-92c2-81df9149c0df None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:30.981172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eee77-ec7d-436c-b3a7-b0bc8995374d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:31.023405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:31.059506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74589829-7822-41f4-bacc-732a61504360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:31.061439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74589829-7822-41f4-bacc-732a61504360 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:31.062007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74589829-7822-41f4-bacc-732a61504360 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:31.062497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74589829-7822-41f4-bacc-732a61504360 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:31.064800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74589829-7822-41f4-bacc-732a61504360 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:31.065365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74589829-7822-41f4-bacc-732a61504360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:31.069830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74589829-7822-41f4-bacc-732a61504360 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:31.089522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 204/410] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:31.098449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a115d4-3900-4c70-adc5-eb0c90ed2c9a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:31.098449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a115d4-3900-4c70-adc5-eb0c90ed2c9a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:31.098449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a115d4-3900-4c70-adc5-eb0c90ed2c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:31.105884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:31.122887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-28a115d4-3900-4c70-adc5-eb0c90ed2c9a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:31.124533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-28a115d4-3900-4c70-adc5-eb0c90ed2c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:31.147550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eff23-cf06-4209-92c2-81df9149c0df None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:31.196748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b0bd5-2166-4f46-9618-8200f5326b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:31.197726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b0bd5-2166-4f46-9618-8200f5326b46 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:31.198910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b0bd5-2166-4f46-9618-8200f5326b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:31.199402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b0bd5-2166-4f46-9618-8200f5326b46 None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:31.202168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b0bd5-2166-4f46-9618-8200f5326b46 None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:31.204011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b0bd5-2166-4f46-9618-8200f5326b46 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:31.209533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 207/411] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:30 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:31.210692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b0bd5-2166-4f46-9618-8200f5326b46 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:31.221492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 205/412] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:31.221492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a115d4-3900-4c70-adc5-eb0c90ed2c9a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:31.378911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b0bd5-2166-4f46-9618-8200f5326b46 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:31.400084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a115d4-3900-4c70-adc5-eb0c90ed2c9a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:31.416165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:31.458873 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206774a2-421a-41ce-9aef-2fd2569540cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:31.459538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206774a2-421a-41ce-9aef-2fd2569540cd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:31.459828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206774a2-421a-41ce-9aef-2fd2569540cd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:31.460111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206774a2-421a-41ce-9aef-2fd2569540cd None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:31.461586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206774a2-421a-41ce-9aef-2fd2569540cd None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:31.463299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206774a2-421a-41ce-9aef-2fd2569540cd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:31.469602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 206/413] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:31 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:31.472182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206774a2-421a-41ce-9aef-2fd2569540cd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:31.477604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea7e1-eb52-4a77-9e6e-e8b32850929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:31.478588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea7e1-eb52-4a77-9e6e-e8b32850929b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:31.479074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea7e1-eb52-4a77-9e6e-e8b32850929b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:31.479550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea7e1-eb52-4a77-9e6e-e8b32850929b None admin] PATH_INFO: `/v3/groups/2d344a51c948420586569fb7b54b400f` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:31.486925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ea7e1-eb52-4a77-9e6e-e8b32850929b None admin] RBAC: Authorizing `identity:get_group(group_id=2d344a51c948420586569fb7b54b400f)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:31.488437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ea7e1-eb52-4a77-9e6e-e8b32850929b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:31.495563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 208/414] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:31 2022] GET /identity/v3/groups/2d344a51c948420586569fb7b54b400f => generated 229 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:31.501900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea7e1-eb52-4a77-9e6e-e8b32850929b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:31.537840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:31.672629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea7e1-eb52-4a77-9e6e-e8b32850929b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:31.723024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4108-8d7a-46d9-a64b-0dcab71b3eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:31.723725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4108-8d7a-46d9-a64b-0dcab71b3eac None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:31.724014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4108-8d7a-46d9-a64b-0dcab71b3eac None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:31.724324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4108-8d7a-46d9-a64b-0dcab71b3eac None admin] PATH_INFO: `/v3/projects/cf1b6fdf131f4a598a5705727a85c56c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:31.725848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4108-8d7a-46d9-a64b-0dcab71b3eac None admin] RBAC: Authorizing `identity:get_project(project_id=cf1b6fdf131f4a598a5705727a85c56c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:31.727406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4108-8d7a-46d9-a64b-0dcab71b3eac None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:31.729748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 209/415] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:31 2022] GET /identity/v3/projects/cf1b6fdf131f4a598a5705727a85c56c => generated 344 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:31.730791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4108-8d7a-46d9-a64b-0dcab71b3eac None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:31.742649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206774a2-421a-41ce-9aef-2fd2569540cd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:31.829152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ad450-f460-4c39-8452-f28cb5eaed13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:31.830899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ad450-f460-4c39-8452-f28cb5eaed13 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:31.831418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ad450-f460-4c39-8452-f28cb5eaed13 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:31.831911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ad450-f460-4c39-8452-f28cb5eaed13 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b/groups/f4037f7fa305455fb12f132d2f0c36af/roles/753d3bbfcdaa4cef86cc8de37ae19906` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:31.837188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ad450-f460-4c39-8452-f28cb5eaed13 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a96496b71e840f8b6e17be47512394b, group_id=f4037f7fa305455fb12f132d2f0c36af, role_id=753d3bbfcdaa4cef86cc8de37ae19906)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:31.839954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ad450-f460-4c39-8452-f28cb5eaed13 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:31.858796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 207/416] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Oct 3 13:16:31 2022] PUT /identity/v3/projects/5a96496b71e840f8b6e17be47512394b/groups/f4037f7fa305455fb12f132d2f0c36af/roles/753d3bbfcdaa4cef86cc8de37ae19906 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:16:31.864184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ad450-f460-4c39-8452-f28cb5eaed13 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:31.898850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:31.942608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4108-8d7a-46d9-a64b-0dcab71b3eac None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:31.957358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:32.011717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58340809-a3df-48f4-83af-c3203030ea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:32.012805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58340809-a3df-48f4-83af-c3203030ea57 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:32.013291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58340809-a3df-48f4-83af-c3203030ea57 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:32.013796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58340809-a3df-48f4-83af-c3203030ea57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:32.026962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58340809-a3df-48f4-83af-c3203030ea57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-58340809-a3df-48f4-83af-c3203030ea57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:32.033573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:32.034538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:32.035339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:32.037819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 210/417] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:16:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:32.039550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949195-dd07-4e69-8a1a-007f4f411108 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:32.040111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949195-dd07-4e69-8a1a-007f4f411108 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:32.040579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949195-dd07-4e69-8a1a-007f4f411108 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:32.042681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 211/418] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:32.043754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3949195-dd07-4e69-8a1a-007f4f411108 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:32.074926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ad450-f460-4c39-8452-f28cb5eaed13 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:32.127191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2e2f7-de23-45c4-ad8e-390e50b1c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:32.128040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e2f7-de23-45c4-ad8e-390e50b1c915 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:32.128419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e2f7-de23-45c4-ad8e-390e50b1c915 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:32.128793 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2e2f7-de23-45c4-ad8e-390e50b1c915 None admin] PATH_INFO: `/v3/projects/cf1b6fdf131f4a598a5705727a85c56c/groups/2d344a51c948420586569fb7b54b400f/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:32.130994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2e2f7-de23-45c4-ad8e-390e50b1c915 None admin] RBAC: Authorizing `identity:create_grant(project_id=cf1b6fdf131f4a598a5705727a85c56c, group_id=2d344a51c948420586569fb7b54b400f, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:32.132976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2e2f7-de23-45c4-ad8e-390e50b1c915 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:32.150869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 208/419] 10.222.0.58 () {58 vars in 1442 bytes} [Mon Oct 3 13:16:31 2022] PUT /identity/v3/projects/cf1b6fdf131f4a598a5705727a85c56c/groups/2d344a51c948420586569fb7b54b400f/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:16:32.152535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1b65e-3b0d-4d60-87f7-38dc095ea5fd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:32.153119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1b65e-3b0d-4d60-87f7-38dc095ea5fd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:32.153763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1b65e-3b0d-4d60-87f7-38dc095ea5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:32.177734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-17d1b65e-3b0d-4d60-87f7-38dc095ea5fd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:32.179253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-17d1b65e-3b0d-4d60-87f7-38dc095ea5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:32.180676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3949195-dd07-4e69-8a1a-007f4f411108 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:32.223937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:32.224937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:32.225483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:32.226030 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:32.226951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:32.227462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:32.229076 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:32.241872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 212/420] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:32.255241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:32.259104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:32.270339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 209/421] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:32.462987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9e081-8c20-4363-abf7-e3d5e85c5670 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:32.535719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c293af-1b0c-4a03-83b4-ea86fc55caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:32.536844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c293af-1b0c-4a03-83b4-ea86fc55caf9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:32.537315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c293af-1b0c-4a03-83b4-ea86fc55caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:32.537839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c293af-1b0c-4a03-83b4-ea86fc55caf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:32.666931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c293af-1b0c-4a03-83b4-ea86fc55caf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:32.669605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c293af-1b0c-4a03-83b4-ea86fc55caf9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:32.672482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:32.828374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1b65e-3b0d-4d60-87f7-38dc095ea5fd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:32.830511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 213/422] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:32.839076 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c293af-1b0c-4a03-83b4-ea86fc55caf9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:32.909857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:33.066802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1b65e-3b0d-4d60-87f7-38dc095ea5fd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:33.103464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c293af-1b0c-4a03-83b4-ea86fc55caf9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:33.107934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:33.133782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beb713-91c7-414e-ad04-071fca40681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:33.134842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb713-91c7-414e-ad04-071fca40681a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:33.135342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb713-91c7-414e-ad04-071fca40681a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:33.135853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75beb713-91c7-414e-ad04-071fca40681a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:33.160737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75beb713-91c7-414e-ad04-071fca40681a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-75beb713-91c7-414e-ad04-071fca40681a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:33.170768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:33.172608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:33.175099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 210/423] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:33.182786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beb713-91c7-414e-ad04-071fca40681a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:33.187151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a847db7-f82a-49c6-8fe1-053f773c22b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:33.188052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a847db7-f82a-49c6-8fe1-053f773c22b1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:33.188340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a847db7-f82a-49c6-8fe1-053f773c22b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:33.188618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a847db7-f82a-49c6-8fe1-053f773c22b1 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:33.190245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a847db7-f82a-49c6-8fe1-053f773c22b1 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:33.191731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a847db7-f82a-49c6-8fe1-053f773c22b1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:33.197835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 214/424] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:32 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:33.198862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a847db7-f82a-49c6-8fe1-053f773c22b1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:33.359887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:33.381008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75beb713-91c7-414e-ad04-071fca40681a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:33.435447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a847db7-f82a-49c6-8fe1-053f773c22b1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:33.461302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:33.462410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:33.462907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:33.463726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:33.464989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:33.465570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:33.468265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:33.501851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 211/425] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:33.508054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:33.526238 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf398885-c92e-445b-a215-3efbf1eb1582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:33.526932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf398885-c92e-445b-a215-3efbf1eb1582 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:33.527205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf398885-c92e-445b-a215-3efbf1eb1582 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:33.527480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf398885-c92e-445b-a215-3efbf1eb1582 None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:33.528808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf398885-c92e-445b-a215-3efbf1eb1582 None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:33.530341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf398885-c92e-445b-a215-3efbf1eb1582 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:33.532291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 215/426] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:33 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:33.538653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf398885-c92e-445b-a215-3efbf1eb1582 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:33.591089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:33.711873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf398885-c92e-445b-a215-3efbf1eb1582 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:33.713581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64222417-113c-48db-92a0-a772a3f0f1eb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:33.757721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb407b9a-8537-4cd6-8fff-51a5cc7be973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:33.758540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb407b9a-8537-4cd6-8fff-51a5cc7be973 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:33.758917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb407b9a-8537-4cd6-8fff-51a5cc7be973 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:33.759323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb407b9a-8537-4cd6-8fff-51a5cc7be973 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:33.760180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb407b9a-8537-4cd6-8fff-51a5cc7be973 None admin] RBAC: Adding query filter params (group.id=f4037f7fa305455fb12f132d2f0c36af, role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=dda4727fec9747b4b20cb990f7305e3d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:33.760602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb407b9a-8537-4cd6-8fff-51a5cc7be973 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:33.762350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb407b9a-8537-4cd6-8fff-51a5cc7be973 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:33.773753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 216/427] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:33 2022] GET /identity/v3/role_assignments?group.id=f4037f7fa305455fb12f132d2f0c36af&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:33.775401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34022ae4-4272-4f41-8bc0-621f87c2b9a6 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:33.775828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34022ae4-4272-4f41-8bc0-621f87c2b9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:33.776198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34022ae4-4272-4f41-8bc0-621f87c2b9a6 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:33.786359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 217/428] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:33.795100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086f17e-6bda-4628-8a56-f935b1ddf7e3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:33.795492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086f17e-6bda-4628-8a56-f935b1ddf7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:33.795757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086f17e-6bda-4628-8a56-f935b1ddf7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:33.809500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5881c-0059-4d13-b20a-7ad840c70ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:33.811152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5881c-0059-4d13-b20a-7ad840c70ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:33.812021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5881c-0059-4d13-b20a-7ad840c70ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:33.812824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5881c-0059-4d13-b20a-7ad840c70ea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:33.838752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f086f17e-6bda-4628-8a56-f935b1ddf7e3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:33.840912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f086f17e-6bda-4628-8a56-f935b1ddf7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:33.899841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:34.006450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 218/429] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:34.028661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5881c-0059-4d13-b20a-7ad840c70ea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:34.030258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5881c-0059-4d13-b20a-7ad840c70ea9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:34.216023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 212/430] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 13:16:34.225539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086f17e-6bda-4628-8a56-f935b1ddf7e3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:34.270876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:34.344582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5881c-0059-4d13-b20a-7ad840c70ea9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:34.366370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086f17e-6bda-4628-8a56-f935b1ddf7e3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:34.415434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e3ffb-303c-461e-8a87-bc233774d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:34.416100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e3ffb-303c-461e-8a87-bc233774d864 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:34.416385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e3ffb-303c-461e-8a87-bc233774d864 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:34.416668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e3ffb-303c-461e-8a87-bc233774d864 None admin] PATH_INFO: `/v3/projects/dda4727fec9747b4b20cb990f7305e3d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:34.418121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e3ffb-303c-461e-8a87-bc233774d864 None admin] RBAC: Authorizing `identity:get_project(project_id=dda4727fec9747b4b20cb990f7305e3d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:34.420160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e3ffb-303c-461e-8a87-bc233774d864 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:34.422550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 219/431] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:34 2022] GET /identity/v3/projects/dda4727fec9747b4b20cb990f7305e3d => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:34.428732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e3ffb-303c-461e-8a87-bc233774d864 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:34.433701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:34.510241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5881c-0059-4d13-b20a-7ad840c70ea9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:34.539811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:34.543958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e3ffb-303c-461e-8a87-bc233774d864 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:34.575066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85030d4a-ecc2-43d4-bea1-af96c0fba655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:34.575838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85030d4a-ecc2-43d4-bea1-af96c0fba655 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:34.576128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85030d4a-ecc2-43d4-bea1-af96c0fba655 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:34.576552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85030d4a-ecc2-43d4-bea1-af96c0fba655 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:34.584994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164fe6-e26d-4e9a-b616-a670c7720148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:34.585678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164fe6-e26d-4e9a-b616-a670c7720148 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:34.585962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164fe6-e26d-4e9a-b616-a670c7720148 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:34.586241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164fe6-e26d-4e9a-b616-a670c7720148 None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:34.587395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3164fe6-e26d-4e9a-b616-a670c7720148 None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:34.588862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3164fe6-e26d-4e9a-b616-a670c7720148 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:34.590543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 220/432] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:34 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:34.592522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85030d4a-ecc2-43d4-bea1-af96c0fba655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:34.597404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164fe6-e26d-4e9a-b616-a670c7720148 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-85030d4a-ecc2-43d4-bea1-af96c0fba655 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:34.613544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:34.615801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 03 13:16:34.618037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:34.625988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 213/433] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:34.627620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85030d4a-ecc2-43d4-bea1-af96c0fba655 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:34.713955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164fe6-e26d-4e9a-b616-a670c7720148 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:34.753152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:34.753812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:34.754093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:34.755692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:34.755692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] RBAC: Adding query filter params (group.id=f4037f7fa305455fb12f132d2f0c36af, role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=dda4727fec9747b4b20cb990f7305e3d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:34.755692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:34.757130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:34.763978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 221/434] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:34 2022] GET /identity/v3/role_assignments?group.id=f4037f7fa305455fb12f132d2f0c36af&scope.project.id=dda4727fec9747b4b20cb990f7305e3d&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:34.808381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85030d4a-ecc2-43d4-bea1-af96c0fba655 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:34.825092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:34.852321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f126db-9b96-4f4e-8bbf-a2ed106b9a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:34.852965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f126db-9b96-4f4e-8bbf-a2ed106b9a55 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:34.853252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f126db-9b96-4f4e-8bbf-a2ed106b9a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:34.853569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f126db-9b96-4f4e-8bbf-a2ed106b9a55 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:34.854289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f126db-9b96-4f4e-8bbf-a2ed106b9a55 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:34.854601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f126db-9b96-4f4e-8bbf-a2ed106b9a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:34.856072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f126db-9b96-4f4e-8bbf-a2ed106b9a55 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:34.866837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 214/435] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:16:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:34.873333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:34.919841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:35.073597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61afb48-65c6-4355-aadf-4a88e7b5eac2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.073597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61afb48-65c6-4355-aadf-4a88e7b5eac2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.073597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61afb48-65c6-4355-aadf-4a88e7b5eac2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.074572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59790b68-4415-47f2-afb2-d864bdb76c71 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:35.077399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 215/436] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:16:35.082098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31713b1e-9a2e-450b-baae-3aafa80bca8b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.082405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31713b1e-9a2e-450b-baae-3aafa80bca8b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.082659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31713b1e-9a2e-450b-baae-3aafa80bca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.110868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-31713b1e-9a2e-450b-baae-3aafa80bca8b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:35.112594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-31713b1e-9a2e-450b-baae-3aafa80bca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:35.141837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65f070-b7ec-4b9c-a862-b9fdb9fc2757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:35.142516 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65f070-b7ec-4b9c-a862-b9fdb9fc2757 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.142794 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65f070-b7ec-4b9c-a862-b9fdb9fc2757 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.143073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65f070-b7ec-4b9c-a862-b9fdb9fc2757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.218078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 216/437] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:35.220418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eadbe7-55ce-479d-8c3d-3d96f2e7a9ca None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.220927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eadbe7-55ce-479d-8c3d-3d96f2e7a9ca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.221422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eadbe7-55ce-479d-8c3d-3d96f2e7a9ca None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.225382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 217/438] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:35.234698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b22a2-afbd-47cf-871e-ddc0863eaeef None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.234698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b22a2-afbd-47cf-871e-ddc0863eaeef None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.235284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b22a2-afbd-47cf-871e-ddc0863eaeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.248367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65f070-b7ec-4b9c-a862-b9fdb9fc2757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:35.252288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65f070-b7ec-4b9c-a862-b9fdb9fc2757 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:35.264066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5a4b22a2-afbd-47cf-871e-ddc0863eaeef None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:35.265321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5a4b22a2-afbd-47cf-871e-ddc0863eaeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:35.321116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:35.386380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 218/439] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:35.442525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 222/440] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:35.448745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b22a2-afbd-47cf-871e-ddc0863eaeef None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:35.488434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:35.501040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65f070-b7ec-4b9c-a862-b9fdb9fc2757 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:35.626782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b22a2-afbd-47cf-871e-ddc0863eaeef None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:35.686008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65f070-b7ec-4b9c-a862-b9fdb9fc2757 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:35.696485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce326269-0b78-47d5-b54d-59a032dac3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:35.697144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326269-0b78-47d5-b54d-59a032dac3ff None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.697523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326269-0b78-47d5-b54d-59a032dac3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.697808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326269-0b78-47d5-b54d-59a032dac3ff None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.699046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce326269-0b78-47d5-b54d-59a032dac3ff None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:35.701051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce326269-0b78-47d5-b54d-59a032dac3ff None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:35.702263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 219/441] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:35 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:35.703065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce326269-0b78-47d5-b54d-59a032dac3ff None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:35.732795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:35.739252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d5439-e35a-4665-b1ff-4b79269bd8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:35.739904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d5439-e35a-4665-b1ff-4b79269bd8bb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.740190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d5439-e35a-4665-b1ff-4b79269bd8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.740461 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d5439-e35a-4665-b1ff-4b79269bd8bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.753089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d5439-e35a-4665-b1ff-4b79269bd8bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-7c9d5439-e35a-4665-b1ff-4b79269bd8bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:35.761892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:35.763784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:35.764734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 223/442] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:16:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:35.765903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d5439-e35a-4665-b1ff-4b79269bd8bb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:35.841883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce326269-0b78-47d5-b54d-59a032dac3ff None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:35.845884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:35.897114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad5fae-c053-4830-9039-e784ccc0dbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:35.897713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5fae-c053-4830-9039-e784ccc0dbdb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.898051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5fae-c053-4830-9039-e784ccc0dbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.898289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad5fae-c053-4830-9039-e784ccc0dbdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.911688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad5fae-c053-4830-9039-e784ccc0dbdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:35.912356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d5439-e35a-4665-b1ff-4b79269bd8bb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-10ad5fae-c053-4830-9039-e784ccc0dbdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:35.917245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:35.918695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 220/443] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:16:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:35.920214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad5fae-c053-4830-9039-e784ccc0dbdb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:35.973400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2b87-75a8-40c3-8d73-7e7989ea068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:35.974556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2b87-75a8-40c3-8d73-7e7989ea068e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:35.975062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2b87-75a8-40c3-8d73-7e7989ea068e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:35.975551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2b87-75a8-40c3-8d73-7e7989ea068e None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:35.977657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e2b87-75a8-40c3-8d73-7e7989ea068e None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:35.980502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e2b87-75a8-40c3-8d73-7e7989ea068e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:35.992062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 224/444] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:35 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:35.992062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2b87-75a8-40c3-8d73-7e7989ea068e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:36.052453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad5fae-c053-4830-9039-e784ccc0dbdb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:36.070095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:36.092171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:36.092845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:36.093064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:36.093338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:36.094056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:36.094367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:36.096626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:36.107667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 221/445] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:36.108877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:36.131698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2b87-75a8-40c3-8d73-7e7989ea068e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:36.153570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:36.180603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:36.181426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:36.181862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:36.182210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:36.182990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:36.183379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:36.185251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:36.196575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 225/446] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:36.197485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:36.264738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038696cf-80c2-4bf8-87a5-26015e3a9c65 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:36.311094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:36.311886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:36.312258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:36.312779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:36.314462 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] RBAC: Adding query filter params (group.id=f4037f7fa305455fb12f132d2f0c36af, role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8, scope.project.id=5a96496b71e840f8b6e17be47512394b) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:36.314915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:36.316920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:36.325598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 222/447] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:36 2022] GET /identity/v3/role_assignments?group.id=f4037f7fa305455fb12f132d2f0c36af&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 611 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:36.327432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:36.342587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14049433-8d66-4695-80e6-7cb4b9b8936d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:36.390127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:36.416411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6899-99f8-4ba8-ab2e-77f564cb8db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:36.417322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6899-99f8-4ba8-ab2e-77f564cb8db5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:36.417836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6899-99f8-4ba8-ab2e-77f564cb8db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:36.418305 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6899-99f8-4ba8-ab2e-77f564cb8db5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:36.501971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:36.526186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37a01-000e-4066-89ad-cdf6d745649f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:36.539357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f6899-99f8-4ba8-ab2e-77f564cb8db5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:36.541056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f6899-99f8-4ba8-ab2e-77f564cb8db5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:36.578473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0722984-5903-4bd6-9186-5bf216a4d988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:36.579106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0722984-5903-4bd6-9186-5bf216a4d988 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:36.580057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0722984-5903-4bd6-9186-5bf216a4d988 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:36.580468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0722984-5903-4bd6-9186-5bf216a4d988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:36.712881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0722984-5903-4bd6-9186-5bf216a4d988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:36.715629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0722984-5903-4bd6-9186-5bf216a4d988 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:36.722740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 226/448] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:36.729442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6899-99f8-4ba8-ab2e-77f564cb8db5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:36.770324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:36.865688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:36.891062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 223/449] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:36.899430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0722984-5903-4bd6-9186-5bf216a4d988 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:36.903180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f6899-99f8-4ba8-ab2e-77f564cb8db5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:36.955611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e348-b2f8-4a66-938b-aa058f0eef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:36.956400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e348-b2f8-4a66-938b-aa058f0eef05 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:36.956771 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e348-b2f8-4a66-938b-aa058f0eef05 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:36.957141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e348-b2f8-4a66-938b-aa058f0eef05 None admin] PATH_INFO: `/v3/projects/5a96496b71e840f8b6e17be47512394b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:36.958555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98e348-b2f8-4a66-938b-aa058f0eef05 None admin] RBAC: Authorizing `identity:get_project(project_id=5a96496b71e840f8b6e17be47512394b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:36.960141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98e348-b2f8-4a66-938b-aa058f0eef05 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:36.962045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 227/450] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:36 2022] GET /identity/v3/projects/5a96496b71e840f8b6e17be47512394b => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:36.968174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e348-b2f8-4a66-938b-aa058f0eef05 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:37.033330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0722984-5903-4bd6-9186-5bf216a4d988 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:37.071560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7d842-a5bb-4094-bc28-0fda0160a0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:37.072291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d842-a5bb-4094-bc28-0fda0160a0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:37.072635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d842-a5bb-4094-bc28-0fda0160a0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:37.072982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7d842-a5bb-4094-bc28-0fda0160a0a9 None admin] PATH_INFO: `/v3/projects/cf1b6fdf131f4a598a5705727a85c56c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:37.074595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7d842-a5bb-4094-bc28-0fda0160a0a9 None admin] RBAC: Authorizing `identity:get_project(project_id=cf1b6fdf131f4a598a5705727a85c56c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:37.076249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7d842-a5bb-4094-bc28-0fda0160a0a9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:37.078194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 224/451] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:16:36 2022] GET /identity/v3/projects/cf1b6fdf131f4a598a5705727a85c56c => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:37.084721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7d842-a5bb-4094-bc28-0fda0160a0a9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:37.092889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:37.104661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e348-b2f8-4a66-938b-aa058f0eef05 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:37.157143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3b679-1623-4d71-9e2e-6a409097a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:37.159779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b679-1623-4d71-9e2e-6a409097a8ec None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:37.160275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b679-1623-4d71-9e2e-6a409097a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:37.160698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:37.161282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e3b679-1623-4d71-9e2e-6a409097a8ec None admin] PATH_INFO: `/v3/groups/f4037f7fa305455fb12f132d2f0c36af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:37.162967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3b679-1623-4d71-9e2e-6a409097a8ec None admin] RBAC: Authorizing `identity:get_group(group_id=f4037f7fa305455fb12f132d2f0c36af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:37.165928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e3b679-1623-4d71-9e2e-6a409097a8ec None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:37.173352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 228/452] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:36 2022] GET /identity/v3/groups/f4037f7fa305455fb12f132d2f0c36af => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:37.176052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3b679-1623-4d71-9e2e-6a409097a8ec None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:37.224773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7d842-a5bb-4094-bc28-0fda0160a0a9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:37.269058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb14938-49c5-4161-8b3a-cc584a5c09a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:37.269944 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb14938-49c5-4161-8b3a-cc584a5c09a6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:37.270289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb14938-49c5-4161-8b3a-cc584a5c09a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:37.270722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb14938-49c5-4161-8b3a-cc584a5c09a6 None admin] PATH_INFO: `/v3/groups/2d344a51c948420586569fb7b54b400f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:37.272439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb14938-49c5-4161-8b3a-cc584a5c09a6 None admin] RBAC: Authorizing `identity:get_group(group_id=2d344a51c948420586569fb7b54b400f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:37.274156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb14938-49c5-4161-8b3a-cc584a5c09a6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:37.275937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 225/453] 10.222.0.58 () {56 vars in 1180 bytes} [Mon Oct 3 13:16:37 2022] GET /identity/v3/groups/2d344a51c948420586569fb7b54b400f => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:37.282663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb14938-49c5-4161-8b3a-cc584a5c09a6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:37.303920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e3b679-1623-4d71-9e2e-6a409097a8ec None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:37.352424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecf51f-9638-41c1-973a-d7b47b523879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:37.353386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecf51f-9638-41c1-973a-d7b47b523879 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:37.353760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecf51f-9638-41c1-973a-d7b47b523879 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:37.354132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecf51f-9638-41c1-973a-d7b47b523879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:37.355370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecf51f-9638-41c1-973a-d7b47b523879 None admin] RBAC: Adding query filter params (group.id=f4037f7fa305455fb12f132d2f0c36af, role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=5a96496b71e840f8b6e17be47512394b) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:37.355872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecf51f-9638-41c1-973a-d7b47b523879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:37.358478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecf51f-9638-41c1-973a-d7b47b523879 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:37.369506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 229/454] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:37 2022] GET /identity/v3/role_assignments?group.id=f4037f7fa305455fb12f132d2f0c36af&scope.project.id=5a96496b71e840f8b6e17be47512394b&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:37.422708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb14938-49c5-4161-8b3a-cc584a5c09a6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:37.452300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:37.466216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbbe139-248e-4e1d-8850-e466568a74ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:37.467054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbe139-248e-4e1d-8850-e466568a74ee None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:37.467423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbe139-248e-4e1d-8850-e466568a74ee None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:37.467792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbbe139-248e-4e1d-8850-e466568a74ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:37.468652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbe139-248e-4e1d-8850-e466568a74ee None admin] RBAC: Adding query filter params (group.id=2d344a51c948420586569fb7b54b400f, role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=cf1b6fdf131f4a598a5705727a85c56c) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:37.469049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbe139-248e-4e1d-8850-e466568a74ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:37.471306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbbe139-248e-4e1d-8850-e466568a74ee None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:37.478480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 226/455] 10.222.0.58 () {56 vars in 1376 bytes} [Mon Oct 3 13:16:37 2022] GET /identity/v3/role_assignments?group.id=2d344a51c948420586569fb7b54b400f&scope.project.id=cf1b6fdf131f4a598a5705727a85c56c&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:40.904192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdc0fc-87c0-4b7e-a849-e5c820dd9ec5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:40.905018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdc0fc-87c0-4b7e-a849-e5c820dd9ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:40.905018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdc0fc-87c0-4b7e-a849-e5c820dd9ec5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:40.906448 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 230/456] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:40.916025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a41da5-dfac-46b4-a12d-39b5c979e461 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:40.917006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a41da5-dfac-46b4-a12d-39b5c979e461 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:40.917536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a41da5-dfac-46b4-a12d-39b5c979e461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:40.946655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-24a41da5-dfac-46b4-a12d-39b5c979e461 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:40.948474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-24a41da5-dfac-46b4-a12d-39b5c979e461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:41.048955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 227/457] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:41.244820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eff3b-4172-4a0d-b903-f94fff358b2a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:41.245523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eff3b-4172-4a0d-b903-f94fff358b2a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:41.246001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eff3b-4172-4a0d-b903-f94fff358b2a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:41.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 231/458] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:41.256771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc915d82-734f-4550-9e72-db50706549b9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:41.257396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc915d82-734f-4550-9e72-db50706549b9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:41.257897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc915d82-734f-4550-9e72-db50706549b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:41.286731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cc915d82-734f-4550-9e72-db50706549b9 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:41.288653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cc915d82-734f-4550-9e72-db50706549b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:41.399829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 228/459] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:41.607368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad4280-f34f-4e8b-800f-6f07c8ed8043 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:41.608029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad4280-f34f-4e8b-800f-6f07c8ed8043 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:41.608532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad4280-f34f-4e8b-800f-6f07c8ed8043 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:41.610483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 232/460] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:41.618525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e9f6d-b370-4526-93ee-8af46e2128ee None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:41.619139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e9f6d-b370-4526-93ee-8af46e2128ee None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:41.619608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e9f6d-b370-4526-93ee-8af46e2128ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:41.654700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-298e9f6d-b370-4526-93ee-8af46e2128ee None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:41.659229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-298e9f6d-b370-4526-93ee-8af46e2128ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:41.671552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad4280-f34f-4e8b-800f-6f07c8ed8043 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:41.729880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:41.732000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:41.805911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 229/461] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:41.807009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e9f6d-b370-4526-93ee-8af46e2128ee None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:41.828430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad4280-f34f-4e8b-800f-6f07c8ed8043 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:41.918290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133493fb-c95b-43fd-8e39-9f8f847874d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:41.918910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133493fb-c95b-43fd-8e39-9f8f847874d9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:41.919168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133493fb-c95b-43fd-8e39-9f8f847874d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:41.919404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133493fb-c95b-43fd-8e39-9f8f847874d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:41.947849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133493fb-c95b-43fd-8e39-9f8f847874d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:41.949330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133493fb-c95b-43fd-8e39-9f8f847874d9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:41.960336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e9f6d-b370-4526-93ee-8af46e2128ee None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-133493fb-c95b-43fd-8e39-9f8f847874d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:41.982996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:41.984195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:41.985336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:41.985336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:41.985336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:41.995184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 233/462] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:41.996784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598119-67af-499b-8b57-8c30020752ed None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:41.997252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598119-67af-499b-8b57-8c30020752ed None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:41.997740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598119-67af-499b-8b57-8c30020752ed None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:41.999262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 234/463] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:42.001959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3598119-67af-499b-8b57-8c30020752ed None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:42.002926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838fa5f7-5863-41dc-b215-b0ce1341d847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:42.003646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fa5f7-5863-41dc-b215-b0ce1341d847 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.004168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fa5f7-5863-41dc-b215-b0ce1341d847 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.004459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838fa5f7-5863-41dc-b215-b0ce1341d847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.019426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fa5f7-5863-41dc-b215-b0ce1341d847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:42.021575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838fa5f7-5863-41dc-b215-b0ce1341d847 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-838fa5f7-5863-41dc-b215-b0ce1341d847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:42.041774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:42.042760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:42.043622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:42.043622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:42.043622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:42.043622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 230/464] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:42.044443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4d1f5-cab5-4410-a5ba-6ec7a0584532 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.044726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4d1f5-cab5-4410-a5ba-6ec7a0584532 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.045002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4d1f5-cab5-4410-a5ba-6ec7a0584532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.069440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-98f4d1f5-cab5-4410-a5ba-6ec7a0584532 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:42.070688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-98f4d1f5-cab5-4410-a5ba-6ec7a0584532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:42.097342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:42.103730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:42.135232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3598119-67af-499b-8b57-8c30020752ed None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:42.144967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 231/465] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:42.146268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4d1f5-cab5-4410-a5ba-6ec7a0584532 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:42.176073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:42.176942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.177376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.177862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.178677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:42.179136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:42.180704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:42.196000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 235/466] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:42.201602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:42.254924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4d1f5-cab5-4410-a5ba-6ec7a0584532 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:42.294756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:42.295329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.295615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.295897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.296577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:42.296886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:42.299962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:42.312785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 232/467] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:42.318460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:42.363210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:42.366348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc16e2-ed1c-4883-9bc9-ecdc599d8de9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:42.431242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:42.431835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.432191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.432479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.434890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:42.447620 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:42.449113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:42.466605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:42.467582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:42.468444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:42.468444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:42.468444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:42.468444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 236/468] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:42.468680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee385c-3d63-404a-b566-9e1cd027eed6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:42.468753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:42.527778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02988b7-1e3b-4822-bb10-228ad3f796d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:42.528405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02988b7-1e3b-4822-bb10-228ad3f796d3 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.528700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02988b7-1e3b-4822-bb10-228ad3f796d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.528989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02988b7-1e3b-4822-bb10-228ad3f796d3 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.529698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02988b7-1e3b-4822-bb10-228ad3f796d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:42.531169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02988b7-1e3b-4822-bb10-228ad3f796d3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:42.561512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 233/469] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:16:42 2022] POST /identity/v3/users => generated 246 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:16:42.561512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02988b7-1e3b-4822-bb10-228ad3f796d3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:42.625395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c679fb-2d0e-4a2e-b548-f5a7d245a857 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:42.681730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609d58-1cfb-452d-8604-6403ade1afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:42.682346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609d58-1cfb-452d-8604-6403ade1afbc None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.682635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609d58-1cfb-452d-8604-6403ade1afbc None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.682916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609d58-1cfb-452d-8604-6403ade1afbc None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.683660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609d58-1cfb-452d-8604-6403ade1afbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:42.684824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:42.688362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609d58-1cfb-452d-8604-6403ade1afbc None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:42.725568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02988b7-1e3b-4822-bb10-228ad3f796d3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:42.726758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 237/470] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:16:42 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:16:42.727773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609d58-1cfb-452d-8604-6403ade1afbc None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:42.779258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305fb58-f600-4b93-9233-e59c774f383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:42.780066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305fb58-f600-4b93-9233-e59c774f383f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.780355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305fb58-f600-4b93-9233-e59c774f383f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.780719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305fb58-f600-4b93-9233-e59c774f383f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.791473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:42.799680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305fb58-f600-4b93-9233-e59c774f383f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:42.802826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305fb58-f600-4b93-9233-e59c774f383f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3305fb58-f600-4b93-9233-e59c774f383f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:42.821756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:42.822973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:42.823964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:42.823964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:42.823964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:42.826089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 234/471] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:42.827773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b52ac7-5291-4c40-96e3-12d4ad59ff84 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.828232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b52ac7-5291-4c40-96e3-12d4ad59ff84 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.828651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b52ac7-5291-4c40-96e3-12d4ad59ff84 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.831249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 235/472] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:42.839964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d5c46-2a84-4d0f-b179-66bd3face472 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.840370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d5c46-2a84-4d0f-b179-66bd3face472 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.840536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d5c46-2a84-4d0f-b179-66bd3face472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.863444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7c8d5c46-2a84-4d0f-b179-66bd3face472 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:42.865119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7c8d5c46-2a84-4d0f-b179-66bd3face472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:42.901714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609d58-1cfb-452d-8604-6403ade1afbc None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:42.974329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:42.975333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:42.975333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:42.975641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:42.976171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:42.976479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:42.977923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:42.994414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 238/473] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:42.995257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:42.997724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 236/474] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:43.001560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d5c46-2a84-4d0f-b179-66bd3face472 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:43.087781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:43.141628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d5c46-2a84-4d0f-b179-66bd3face472 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:43.194015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ddc5-edc0-4764-b96e-e6c69adfab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:43.194947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ddc5-edc0-4764-b96e-e6c69adfab1b None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:43.195451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ddc5-edc0-4764-b96e-e6c69adfab1b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:43.195909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ddc5-edc0-4764-b96e-e6c69adfab1b None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:43.196326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b41e61-76f9-425f-8680-f424179e363e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:43.197391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403ddc5-edc0-4764-b96e-e6c69adfab1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:43.199544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403ddc5-edc0-4764-b96e-e6c69adfab1b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:43.201850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:43.259879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:43.261106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:43.262146 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:43.262659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:43.263943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:43.264463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:43.267359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:43.273735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 237/475] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:16:42 2022] POST /identity/v3/users => generated 248 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:16:43.289531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 239/476] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:43.295503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ddc5-edc0-4764-b96e-e6c69adfab1b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:43.317204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:43.463632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c6478-d431-4fda-b345-e5dbaadd636a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:43.513605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:43.513605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:43.513605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:43.513605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:43.514175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:43.514175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:43.522763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.sql.core [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 13:16:43.522763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 03 13:16:43.522763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [parameters: {'id': 'baac1f4cf035432da7ca6e19b74ee78c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 03 13:16:43.522763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81423) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application conn.query(q) Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result.read() Oct 03 13:16:43.549286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return ref.to_dict() Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 13:16:43.550966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application next(self.gen) Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application yield resource Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application next(self.gen) Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise self.value Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application session.commit() Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 13:16:43.552760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.session.flush() Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application compat.raise_( Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 13:16:43.556265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 13:16:43.559602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application conn.query(q) Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result.read() Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 13:16:43.562805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [parameters: {'id': 'baac1f4cf035432da7ca6e19b74ee78c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:43.572281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 03 13:16:43.577859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:43.580396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 240/477] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Oct 3 13:16:43 2022] POST /identity/v3/roles => generated 144 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:43.580396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:43.580800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ddc5-edc0-4764-b96e-e6c69adfab1b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:43.585408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:43.622667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b726b-28e0-4936-99eb-fd92f10ec3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:43.623310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b726b-28e0-4936-99eb-fd92f10ec3d2 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:43.623601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b726b-28e0-4936-99eb-fd92f10ec3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:43.624651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b726b-28e0-4936-99eb-fd92f10ec3d2 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:43.625421 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b726b-28e0-4936-99eb-fd92f10ec3d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:43.627027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b726b-28e0-4936-99eb-fd92f10ec3d2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:43.645435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:43.664092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 238/478] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:16:43 2022] POST /identity/v3/users => generated 249 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:16:43.822144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c4403-96b9-4a0f-b6e1-4fa4099f0c05 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:43.887906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5babd93-43d6-4b86-a4c0-b0a30372d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:43.889592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5babd93-43d6-4b86-a4c0-b0a30372d395 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:43.890317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5babd93-43d6-4b86-a4c0-b0a30372d395 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:43.891010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5babd93-43d6-4b86-a4c0-b0a30372d395 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:43.912844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5babd93-43d6-4b86-a4c0-b0a30372d395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-e5babd93-43d6-4b86-a4c0-b0a30372d395 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:43.925037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:43.926798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:43.933664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 241/479] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:16:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:43.940778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b726b-28e0-4936-99eb-fd92f10ec3d2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:44.130241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b726b-28e0-4936-99eb-fd92f10ec3d2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:44.176292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfe0ac-2c1b-4f9c-a20b-0ea70d109d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:44.177321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfe0ac-2c1b-4f9c-a20b-0ea70d109d0d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:44.177807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfe0ac-2c1b-4f9c-a20b-0ea70d109d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:44.178261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfe0ac-2c1b-4f9c-a20b-0ea70d109d0d None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:44.179126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfe0ac-2c1b-4f9c-a20b-0ea70d109d0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:44.179610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfe0ac-2c1b-4f9c-a20b-0ea70d109d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:44.181227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfe0ac-2c1b-4f9c-a20b-0ea70d109d0d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:44.192397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 239/480] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:45.744555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a79f-adc3-4ad1-839c-d582f8ea2bdc None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:45.746224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a79f-adc3-4ad1-839c-d582f8ea2bdc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:45.746914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058a79f-adc3-4ad1-839c-d582f8ea2bdc None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:45.750046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 242/481] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:45.763957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7379bbc-c7d6-4636-aec5-f20f47ceaebf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:45.765432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7379bbc-c7d6-4636-aec5-f20f47ceaebf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:45.766196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7379bbc-c7d6-4636-aec5-f20f47ceaebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:45.795088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a7379bbc-c7d6-4636-aec5-f20f47ceaebf None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:45.796337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a7379bbc-c7d6-4636-aec5-f20f47ceaebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:45.865311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:45.968117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 240/482] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:46.041158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfc725-66b7-4061-b0df-485604fde1ea None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:46.041784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfc725-66b7-4061-b0df-485604fde1ea None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:46.043996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfc725-66b7-4061-b0df-485604fde1ea None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:46.046954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 243/483] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:46.056357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e953-fde0-47b4-b726-8e762354a3f2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:46.056855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e953-fde0-47b4-b726-8e762354a3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:46.057357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e953-fde0-47b4-b726-8e762354a3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:46.093375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-64d6e953-fde0-47b4-b726-8e762354a3f2 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:46.095512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-64d6e953-fde0-47b4-b726-8e762354a3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:46.271143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 241/484] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:46.579744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfc725-66b7-4061-b0df-485604fde1ea None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:46.748902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f4d04-083b-47d5-b76e-ceec78b2925b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:46.749633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f4d04-083b-47d5-b76e-ceec78b2925b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:46.750123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f4d04-083b-47d5-b76e-ceec78b2925b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:46.753413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 242/485] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:46.761251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef612d-85c5-45d9-9f1c-fb357b3750ad None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:46.761904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef612d-85c5-45d9-9f1c-fb357b3750ad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:46.762380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef612d-85c5-45d9-9f1c-fb357b3750ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:46.776669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:46.785735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-67ef612d-85c5-45d9-9f1c-fb357b3750ad None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:46.787701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-67ef612d-85c5-45d9-9f1c-fb357b3750ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:46.812075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfc725-66b7-4061-b0df-485604fde1ea None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:46.873100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 243/486] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 03 13:16:46.878466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef612d-85c5-45d9-9f1c-fb357b3750ad None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:46.889158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f8460-e5d7-4729-b66a-4e6e4d1650bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:46.890372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f8460-e5d7-4729-b66a-4e6e4d1650bf None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:46.890891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f8460-e5d7-4729-b66a-4e6e4d1650bf None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:46.891432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f8460-e5d7-4729-b66a-4e6e4d1650bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:46.932687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f8460-e5d7-4729-b66a-4e6e4d1650bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-862f8460-e5d7-4729-b66a-4e6e4d1650bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:46.956324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:46.957309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:46.958157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:46.960992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 244/487] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:46.967849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f8460-e5d7-4729-b66a-4e6e4d1650bf None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:46.968854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:47.169244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f8460-e5d7-4729-b66a-4e6e4d1650bf None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:47.212269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef612d-85c5-45d9-9f1c-fb357b3750ad None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:47.235850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:47.236901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.237398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.237914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.239149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:47.239709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:47.242393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:47.257946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 245/488] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:16:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:47.259470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:47.277337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9f665-114e-49f0-be4a-15a632d363e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:47.279351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:47.279638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9f665-114e-49f0-be4a-15a632d363e9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.280490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9f665-114e-49f0-be4a-15a632d363e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.281179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9f665-114e-49f0-be4a-15a632d363e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.302828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9f665-114e-49f0-be4a-15a632d363e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a5b9f665-114e-49f0-be4a-15a632d363e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:47.314297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:47.316328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:47.318254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:47.321965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 244/489] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:47.324218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725480-770b-48fd-bf21-b2979042361f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.324803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725480-770b-48fd-bf21-b2979042361f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.325310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6725480-770b-48fd-bf21-b2979042361f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.331987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 245/490] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:47.331987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6725480-770b-48fd-bf21-b2979042361f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:47.456368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c65d7-77ad-4c45-a58f-4df57deb1aa8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:47.464007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:47.488775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6725480-770b-48fd-bf21-b2979042361f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:47.524052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14824843-8d9c-4dba-93cc-042f7f35b0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:47.525097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824843-8d9c-4dba-93cc-042f7f35b0dc None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.525622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824843-8d9c-4dba-93cc-042f7f35b0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.526103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14824843-8d9c-4dba-93cc-042f7f35b0dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.541031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019101f-35e3-489e-8ea6-14ea8d286011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:47.542696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019101f-35e3-489e-8ea6-14ea8d286011 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.543464 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019101f-35e3-489e-8ea6-14ea8d286011 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.544026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019101f-35e3-489e-8ea6-14ea8d286011 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.544528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14824843-8d9c-4dba-93cc-042f7f35b0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-14824843-8d9c-4dba-93cc-042f7f35b0dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:47.556059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:47.557746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:47.559410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 246/491] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:47.560696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14824843-8d9c-4dba-93cc-042f7f35b0dc None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:47.568600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019101f-35e3-489e-8ea6-14ea8d286011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:47.568600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019101f-35e3-489e-8ea6-14ea8d286011 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0019101f-35e3-489e-8ea6-14ea8d286011 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:47.586597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:47.596827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:47.600516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:47.600516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:47.600516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:47.600516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 246/492] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:47.600516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d9e48-52b7-4722-bf35-68cfdef5eed7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.600516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d9e48-52b7-4722-bf35-68cfdef5eed7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.600516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d9e48-52b7-4722-bf35-68cfdef5eed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.612696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aa1d9e48-52b7-4722-bf35-68cfdef5eed7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:47.614028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aa1d9e48-52b7-4722-bf35-68cfdef5eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:47.694488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:47.722707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 247/493] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:47.726611 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d9e48-52b7-4722-bf35-68cfdef5eed7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:47.771220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14824843-8d9c-4dba-93cc-042f7f35b0dc None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:47.815426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:47.844847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:47.846287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.846802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.847419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.849057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:47.849982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:47.853178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:47.868807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d9e48-52b7-4722-bf35-68cfdef5eed7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:47.870624 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 247/494] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:16:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:47.872501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:47.911417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87c19a-cdbc-48ae-abeb-1a7cc1e02304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:47.912179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87c19a-cdbc-48ae-abeb-1a7cc1e02304 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.912572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87c19a-cdbc-48ae-abeb-1a7cc1e02304 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.912911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87c19a-cdbc-48ae-abeb-1a7cc1e02304 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.914575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87c19a-cdbc-48ae-abeb-1a7cc1e02304 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:47.915337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87c19a-cdbc-48ae-abeb-1a7cc1e02304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:47.918273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87c19a-cdbc-48ae-abeb-1a7cc1e02304 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:47.934899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 248/495] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:16:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:47.937476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3254d-225a-4594-b803-0a8e0cbed197 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:47.937956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3254d-225a-4594-b803-0a8e0cbed197 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:47.938428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3254d-225a-4594-b803-0a8e0cbed197 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:47.940720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 249/496] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:47.941925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3254d-225a-4594-b803-0a8e0cbed197 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:48.012905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef05f6-ba0d-46c5-a929-0ed1f7c7f6be None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.054837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:48.062510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.063256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.063733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.064224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.064911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:48.065219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:48.066683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:48.081649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 248/497] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:48.082905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:48.120019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3254d-225a-4594-b803-0a8e0cbed197 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.161576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:48.174418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b1e73-81cf-457f-a0a8-ae97c3f5d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.175182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b1e73-81cf-457f-a0a8-ae97c3f5d06b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.175554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b1e73-81cf-457f-a0a8-ae97c3f5d06b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.175932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b1e73-81cf-457f-a0a8-ae97c3f5d06b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.190353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b1e73-81cf-457f-a0a8-ae97c3f5d06b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:48.192087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b1e73-81cf-457f-a0a8-ae97c3f5d06b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:48.195009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6be26c-67e5-4a18-b391-540bd5fb1e9b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cc3b1e73-81cf-457f-a0a8-ae97c3f5d06b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:48.209536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:48.210927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:48.212054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:48.212054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:48.212054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:48.212054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 250/498] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:48.212054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0815-4e54-4415-aa2e-62d0135a37af None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.212367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0815-4e54-4415-aa2e-62d0135a37af None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.212433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a0815-4e54-4415-aa2e-62d0135a37af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.231688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a55a0815-4e54-4415-aa2e-62d0135a37af None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:48.232907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a55a0815-4e54-4415-aa2e-62d0135a37af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:48.245894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.246975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.247264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.247636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.267749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:48.270323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:48.290319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:48.291356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:48.292959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:48.292959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:48.292959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:48.292959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 249/499] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:48.292959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:48.314212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 251/500] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:48.315082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a0815-4e54-4415-aa2e-62d0135a37af None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:48.363308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:48.445359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5b6c8-bb0c-4786-8fdb-7c4d4f882db4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.464692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a0815-4e54-4415-aa2e-62d0135a37af None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.476744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:48.486324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6d621-da74-450e-8a81-ba502106a600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.487233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6d621-da74-450e-8a81-ba502106a600 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.487548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6d621-da74-450e-8a81-ba502106a600 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.487811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6d621-da74-450e-8a81-ba502106a600 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.501178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6d621-da74-450e-8a81-ba502106a600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2dd6d621-da74-450e-8a81-ba502106a600 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:48.507592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:48.508538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 250/501] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:48.511132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6d621-da74-450e-8a81-ba502106a600 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:48.522884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.524064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.524597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.525129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.540232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:48.541861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:16:48.557644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:48.562472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 252/502] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:16:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:48.564117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:48.676621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6d621-da74-450e-8a81-ba502106a600 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.679698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:48.686138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ae88-98c8-4e0f-99e9-d19927065de0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.726323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.727199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.727553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.727917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.728815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:48.729256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:48.731083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:48.731278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.731876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.732162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.732440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.733121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:48.733421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:48.735011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:48.746906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 253/503] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:48.748082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:48.753293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 251/504] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:48.754307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:48.797071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:48.887317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f2dd3-c27e-48ea-bc06-6c686eef7aa6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.895907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc29ed8-f7a1-4c5d-99e6-ff80de63eab5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:48.942845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.943616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.944010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.944337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.945066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:48.945443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:48.947052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:48.950021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:48.951069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:48.951563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:48.952032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:48.953333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:48.953991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:48.956599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 254/505] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:16:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:48.956985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:48.957758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.022148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 252/506] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:16:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:49.023371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.046256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:49.082091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:49.094232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88831a-dc32-42ef-bfcf-4da5609f8310 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:49.133719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:49.134363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:49.134648 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:49.134926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:49.143061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb057ad9-32c8-4e07-8d57-5425c025dc9a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:49.146365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:49.147860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:49.163208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:49.164283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:49.165165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:49.165165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:49.165165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:49.165165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 255/507] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:49.165165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.200426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:49.201088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:49.201365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:49.201671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:49.211595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:49.213294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:16:49.228425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:49.230180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 253/508] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:49.231713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.290543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8b700-7883-463f-b0a1-9e02aed1445a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:49.332257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:49.332836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:49.333121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:49.333406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:49.343039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:49.344881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:16:49.357960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:49.359505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 256/509] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:49.361783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.366833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:49.369644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:49.400290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b16cd-50b4-4324-8dfa-0cd4a6204a73 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:49.444350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:49.445019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:49.445306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:49.445619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:49.458432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:49.459958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:49.475141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:49.477008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:49.479241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:49.479241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:49.479241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:49.479241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 254/510] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:49.479241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.540177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ccdc46-3ee8-4207-b7e0-307f4c35a456 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:49.586438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94031372-04f8-47c3-b229-c94e697fcc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:49.586972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94031372-04f8-47c3-b229-c94e697fcc2b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:49.587257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94031372-04f8-47c3-b229-c94e697fcc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:49.587540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94031372-04f8-47c3-b229-c94e697fcc2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:49.628195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d9280-e9b2-4f1a-9100-08daafdb48f3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:49.671630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:49.678961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:49.679589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:49.679879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:49.680153 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:49.680806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:49.681111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:49.682570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:49.683983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94031372-04f8-47c3-b229-c94e697fcc2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:49.686663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94031372-04f8-47c3-b229-c94e697fcc2b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:49.690683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:49.698406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 255/511] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:49.699288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.834340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 257/512] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:49.835294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94031372-04f8-47c3-b229-c94e697fcc2b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.850485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698b90-145c-445d-b237-5a24eb2fb873 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:49.895067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:49.895696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:49.895979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:49.896865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:49.897006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:49.897288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:49.898817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:49.918829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 256/513] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Oct 3 13:16:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:49.919702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:49.959539 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94031372-04f8-47c3-b229-c94e697fcc2b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:49.961294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:50.001557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:50.007473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:50.008184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:50.008476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:50.008755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:50.009576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:50.009892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:50.011371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:50.064808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 258/514] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Oct 3 13:16:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:50.064808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:50.083895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521040dc-ea22-4372-9e4c-6ffbb95a0354 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:50.148258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:50.149142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:50.149657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:50.150104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:50.151201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:50.151576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:50.153982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:50.179942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 257/515] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:50.179942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:50.223045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e17675-9705-4fa7-8f1b-1a6bbe045112 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:50.269379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b511a-cc2d-4f4e-a6da-1515469c6944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:50.270297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b511a-cc2d-4f4e-a6da-1515469c6944 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:50.270764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b511a-cc2d-4f4e-a6da-1515469c6944 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:50.271117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b511a-cc2d-4f4e-a6da-1515469c6944 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:50.271876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b511a-cc2d-4f4e-a6da-1515469c6944 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:50.273758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b511a-cc2d-4f4e-a6da-1515469c6944 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:50.303668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 259/516] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:16:50 2022] POST /identity/v3/users => generated 247 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:16:50.304738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b511a-cc2d-4f4e-a6da-1515469c6944 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:50.325261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90f2e-2038-4a93-bf5c-de1ad904d266 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:50.347886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:50.352297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:50.395582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:50.396539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:50.396737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:50.396878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:50.411211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:50.412641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:50.431484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:50.434730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:50.436396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:50.436396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:50.436396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:50.436396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 258/517] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:50.436396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:50.545416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b511a-cc2d-4f4e-a6da-1515469c6944 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:50.590297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09613042-78c9-4866-b935-e08eb948c576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:50.590932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09613042-78c9-4866-b935-e08eb948c576 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:50.591062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b2ac5-0616-4b8c-9f5f-5323f2336b3f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:50.591261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09613042-78c9-4866-b935-e08eb948c576 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:50.591490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09613042-78c9-4866-b935-e08eb948c576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:50.648560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e303e-e3e3-4985-9815-aa75c0708b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:50.649673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e303e-e3e3-4985-9815-aa75c0708b85 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:50.650166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e303e-e3e3-4985-9815-aa75c0708b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:50.650619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e303e-e3e3-4985-9815-aa75c0708b85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:50.691379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:50.692447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09613042-78c9-4866-b935-e08eb948c576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:50.695329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09613042-78c9-4866-b935-e08eb948c576 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:50.724419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:50.752622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e303e-e3e3-4985-9815-aa75c0708b85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:50.754513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e303e-e3e3-4985-9815-aa75c0708b85 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:50.858575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 260/518] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:50.858575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09613042-78c9-4866-b935-e08eb948c576 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:50.928204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 259/519] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:50.928204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e303e-e3e3-4985-9815-aa75c0708b85 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:50.980773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09613042-78c9-4866-b935-e08eb948c576 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:51.000949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:51.023398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:51.024054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:51.024339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:51.024620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:51.034017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:51.035501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:51.041162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:16:51.049388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:51.050422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 13:16:51.051364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:51.055531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 261/520] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:51.056524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:51.109121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e303e-e3e3-4985-9815-aa75c0708b85 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:51.174020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:51.175073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:51.175555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:51.175850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:51.176949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:51.177526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:51.180395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:51.209128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 260/521] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:51.209128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:51.240605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f0e07-023e-4a76-955e-553199a4a851 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:51.304767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:51.305892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:51.306652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:51.307188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:51.327622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:51.331093 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:51.363399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:51.365356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:51.368026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:51.368026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:51.368026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:51.368026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 262/522] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:51.368026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:51.404083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7bb3b-b78e-4217-ae64-10d341c4e900 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:51.418799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:51.445122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:51.460656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:51.461420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:51.461866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:51.462230 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:51.478708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:51.480241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:51.499556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:51.502484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:51.504567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:51.504567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:51.504567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:51.506260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 261/523] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:51.507004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:51.557869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6908951-b5c8-4142-bc8a-bfd74a7a0f4a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:51.612975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:51.613942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:51.614324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:51.614779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:51.616046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:51.616422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:51.618579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:51.657495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 263/524] 10.222.0.58 () {56 vars in 1139 bytes} [Mon Oct 3 13:16:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:51.657495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:51.687896 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edbf52-7ecd-41cd-ad22-df75bb8137e7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:51.742569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:51.743507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:51.743920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:51.744423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:51.753575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:51.754994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:16:51.773758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:51.774759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:51.775711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 262/525] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:51.776205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:51.794046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:51.811660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a00f-2759-406a-b82e-9d69d8f293c6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:51.840649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:51.868866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:51.869546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:51.869829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:51.870107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:51.870818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:51.871068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:51.872487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:51.886360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 264/526] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:51.891833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:51.956201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3a2f0-0c0a-4622-b41c-929cc8866fdd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:52.009605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:52.010557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:52.010784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:52.011233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:52.012247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:52.012531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:52.014993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:52.025670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94b09d-fcb9-464e-a1e6-5d090a6c981e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:52.034156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 263/527] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:52.035599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:52.074772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625ed99-490a-492b-b2b3-4dab3dbbfc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:52.075743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ed99-490a-492b-b2b3-4dab3dbbfc1d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:52.076205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ed99-490a-492b-b2b3-4dab3dbbfc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:52.076606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625ed99-490a-492b-b2b3-4dab3dbbfc1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:52.106847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:52.190726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:52.199442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625ed99-490a-492b-b2b3-4dab3dbbfc1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:52.201056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625ed99-490a-492b-b2b3-4dab3dbbfc1d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:52.201725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd932c-0708-4812-b963-057c83cfbda0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:52.243904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:52.244526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:52.244827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:52.245070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:52.245880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:52.246187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:52.247619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:52.277262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 264/528] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:52.277262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:52.357546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 265/529] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:16:52.357546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625ed99-490a-492b-b2b3-4dab3dbbfc1d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:52.410061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:52.420894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea78f7-2b45-4180-9bbb-1f952517aa56 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:52.464808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:52.465631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:52.465908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:52.466187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:52.474958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:52.476456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:52.480466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625ed99-490a-492b-b2b3-4dab3dbbfc1d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:16:52.489958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:52.490901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:52.491758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 265/530] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:52.492773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:52.523797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:52.551683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:52.553024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:52.553502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:52.553974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:52.563711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:52.565407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:16:52.580609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:52.581585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:52.582420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 266/531] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:52.582891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:52.675693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb4fb2-c92e-4b02-b7aa-83b4ef04b97b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:52.717932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:52.718594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:52.718866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:52.719638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:52.719986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c3ece3405a2b43288136ddb79b84a106) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:52.720404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:52.721782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:52.728465 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 266/532] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:52 2022] GET /identity/v3/role_assignments?user.id=c3ece3405a2b43288136ddb79b84a106&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:52.729944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:52.766957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5d38b-504a-4a6a-bf16-b14978d156ba None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:52.789016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:52.866578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:52.868582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:52.869437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:52.870573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:52.900680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:52.903360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:52.920187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:52.936577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:52.937633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:52.939319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:52.939319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:52.939319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:52.947124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 267/533] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:52.948336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:52.964962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ff261-0c17-4bc8-8241-16457e50a07e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:53.031116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a6edc-ca73-4b68-800f-b090d4eae55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:53.033504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a6edc-ca73-4b68-800f-b090d4eae55e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:53.033504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a6edc-ca73-4b68-800f-b090d4eae55e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:53.033956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a6edc-ca73-4b68-800f-b090d4eae55e None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:53.035130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a6edc-ca73-4b68-800f-b090d4eae55e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:53.035709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a6edc-ca73-4b68-800f-b090d4eae55e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:53.038643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a6edc-ca73-4b68-800f-b090d4eae55e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:53.065574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 267/534] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:53.067925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b875-27be-4963-a2f9-6624b1d892ab None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:53.068507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b875-27be-4963-a2f9-6624b1d892ab None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:53.068992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b875-27be-4963-a2f9-6624b1d892ab None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:53.071657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 268/535] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:53.073022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420b875-27be-4963-a2f9-6624b1d892ab None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:53.137267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e81f8a-ae37-4d95-8a21-386632abf114 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:53.197686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:53.198597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:53.198885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:53.199164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:53.200102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:53.200408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:53.202046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:53.230222 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 268/536] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:53.232004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:53.251454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420b875-27be-4963-a2f9-6624b1d892ab None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:53.271886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:53.311530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fa21d-c10b-4873-940f-e17ff2e52b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:53.312571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fa21d-c10b-4873-940f-e17ff2e52b32 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:53.313099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fa21d-c10b-4873-940f-e17ff2e52b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:53.313623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fa21d-c10b-4873-940f-e17ff2e52b32 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:53.314816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fa21d-c10b-4873-940f-e17ff2e52b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:53.315349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fa21d-c10b-4873-940f-e17ff2e52b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:53.317961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fa21d-c10b-4873-940f-e17ff2e52b32 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:53.331976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:53.356916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 269/537] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:53.359377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e414a1-6952-4775-a79c-644338bfdfce None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:53.359914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e414a1-6952-4775-a79c-644338bfdfce None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:53.360391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e414a1-6952-4775-a79c-644338bfdfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:53.393005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f2e414a1-6952-4775-a79c-644338bfdfce None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:53.395134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f2e414a1-6952-4775-a79c-644338bfdfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:53.407255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a364cd0-4925-4861-9b95-dfcb177c711d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:53.461792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:53.462444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:53.462727 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:53.463004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:53.463778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=464190e0ccaf4f82942f518df1bf6c03) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:53.464093 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:53.465699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:53.475983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 269/538] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:53 2022] GET /identity/v3/role_assignments?user.id=464190e0ccaf4f82942f518df1bf6c03&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:53.477971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:53.531164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 270/539] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:53.533894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e414a1-6952-4775-a79c-644338bfdfce None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:53.671174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:53.715593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086aa24-8830-4584-9a3a-9f62d5ebad19 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:53.772908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e414a1-6952-4775-a79c-644338bfdfce None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:53.787448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:53.788489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:53.789031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:53.789552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:53.790607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=8fa402682fd24228a77801edf1bb4d59) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:53.791134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:53.792916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:53.806942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 270/540] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:53 2022] GET /identity/v3/role_assignments?user.id=8fa402682fd24228a77801edf1bb4d59&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:53.835931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:53.836973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:53.837526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:53.838000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:53.841015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:53.853406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:53.855594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:16:53.882324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:53.883925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:53.885552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:16:53.885552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:16:53.885552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:16:53.885552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:16:53.885552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:16:53.885552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:16:53.885552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:53.885552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:53.894132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 271/541] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:53.900631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:54.081592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:54.087800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1fab-a42f-431e-adc4-3cd7ce7544c9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:54.137018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:54.137708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:54.139184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:54.139502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:54.140265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:54.140578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:54.142912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:54.171992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 271/542] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:16:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:54.180768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:54.202852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:54.347633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee5bc0e-c3c9-45c4-bd9d-8d232c97abfd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:54.414476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac877c-6a5a-433a-8dd2-406cd17da548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:54.416096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac877c-6a5a-433a-8dd2-406cd17da548 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:54.416863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac877c-6a5a-433a-8dd2-406cd17da548 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:54.417561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac877c-6a5a-433a-8dd2-406cd17da548 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:54.440360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac877c-6a5a-433a-8dd2-406cd17da548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-54ac877c-6a5a-433a-8dd2-406cd17da548 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:54.450373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:54.452074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:54.454282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:54.459020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 272/543] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:54.466195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e840944-9e07-488c-a517-defb7876c692 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:54.468154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac877c-6a5a-433a-8dd2-406cd17da548 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:54.588115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:54.595882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:54.595882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:54.595882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:54.596353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=cf865cb3c7fc432884e4e308c397c83d) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:54.596353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:54.597905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:54.613713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 272/544] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:54 2022] GET /identity/v3/role_assignments?user.id=cf865cb3c7fc432884e4e308c397c83d&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 259 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:54.624692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:54.757847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac877c-6a5a-433a-8dd2-406cd17da548 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:54.815491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:54.816128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:54.816404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:54.816683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:54.817376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:54.817717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:54.819141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:54.835181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 273/545] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:16:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:54.841710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:55.008700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:55.083913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772ba33-01cd-4968-94f5-dc1b3280617b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:55.149783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:55.152068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:55.152865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:55.153673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:55.175294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:55.178607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:55.208483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:55.210223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:55.211974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:55.211974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:55.211974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:55.215591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 273/546] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:16:55.225439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:55.397640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf5889-8b73-4498-8095-c1cdddaca8ef None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:55.409853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:55.449492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:55.450193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:55.450474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:55.451250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:55.451491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:55.451770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:55.453115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:55.473758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 274/547] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:55.479910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:55.631218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1f4ca-ce9e-4170-824e-c62111cdb3c9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:55.659308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:55.679950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5376d3-18b2-45aa-aab5-4a2736c9e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:55.680606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5376d3-18b2-45aa-aab5-4a2736c9e2fa None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:55.680901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5376d3-18b2-45aa-aab5-4a2736c9e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:55.681169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5376d3-18b2-45aa-aab5-4a2736c9e2fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:55.690665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5376d3-18b2-45aa-aab5-4a2736c9e2fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:55.692178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5376d3-18b2-45aa-aab5-4a2736c9e2fa None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5e5376d3-18b2-45aa-aab5-4a2736c9e2fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:16:55.704567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:55.706545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:55.709260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 274/548] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:16:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:55.713537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:55.853242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bec9e7-4add-45bc-9404-225241761037 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:55.853740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bec9e7-4add-45bc-9404-225241761037 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:55.853923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bec9e7-4add-45bc-9404-225241761037 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:55.855467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 275/549] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:55.863926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfd9f5-7923-4629-8667-4aa98d16df78 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:55.864427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfd9f5-7923-4629-8667-4aa98d16df78 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:55.864713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfd9f5-7923-4629-8667-4aa98d16df78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:55.901482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-11cfd9f5-7923-4629-8667-4aa98d16df78 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:55.914723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-11cfd9f5-7923-4629-8667-4aa98d16df78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:55.995467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929513c5-3a0e-466c-9983-dd64128fc7a3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:56.097312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:56.098400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:56.098865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:56.099329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:56.100604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:56.101123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:56.112473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:56.155521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 276/550] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:56.170944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 275/551] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Oct 3 13:16:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:56.178936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfd9f5-7923-4629-8667-4aa98d16df78 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:56.470794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfd9f5-7923-4629-8667-4aa98d16df78 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:56.516708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:56.597686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769160d0-94f6-48f5-b0c9-aa2c26fb2b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:56.598923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769160d0-94f6-48f5-b0c9-aa2c26fb2b24 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:56.599511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769160d0-94f6-48f5-b0c9-aa2c26fb2b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:56.599731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769160d0-94f6-48f5-b0c9-aa2c26fb2b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:56.679617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:56.787451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769160d0-94f6-48f5-b0c9-aa2c26fb2b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:56.790651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769160d0-94f6-48f5-b0c9-aa2c26fb2b24 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:56.805102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:56.999023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb3b07-8595-4f38-b4f2-07e5dbc0b798 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:57.040732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 277/552] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 864 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 03 13:16:57.042870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136b55-7731-41a8-bfa0-f72e80f2dfee None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:57.043345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136b55-7731-41a8-bfa0-f72e80f2dfee None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:57.043825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4136b55-7731-41a8-bfa0-f72e80f2dfee None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:57.046802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf793f-d363-4632-8703-8bcd32df3f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:57.047070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 278/553] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:57.049066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4136b55-7731-41a8-bfa0-f72e80f2dfee None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:57.051574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf793f-d363-4632-8703-8bcd32df3f36 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:57.051866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf793f-d363-4632-8703-8bcd32df3f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:57.052154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf793f-d363-4632-8703-8bcd32df3f36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:57.067357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf793f-d363-4632-8703-8bcd32df3f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-70cf793f-d363-4632-8703-8bcd32df3f36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:57.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:57.081074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:57.082855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:57.088407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 276/554] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:57.089404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:57.092631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fb204-f018-41ae-a0df-5520c235aae4 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:57.093213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fb204-f018-41ae-a0df-5520c235aae4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:57.095028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fb204-f018-41ae-a0df-5520c235aae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:57.118914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0c0fb204-f018-41ae-a0df-5520c235aae4 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:57.120507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0c0fb204-f018-41ae-a0df-5520c235aae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:57.213080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 277/555] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:57.214304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fb204-f018-41ae-a0df-5520c235aae4 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:57.238158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:57.356716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fb204-f018-41ae-a0df-5520c235aae4 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:57.357681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4136b55-7731-41a8-bfa0-f72e80f2dfee None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:57.397910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:57.399188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:57.399664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:57.400121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:57.401736 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:57.402252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:57.404192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:57.418449 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 278/556] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:16:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:57.427339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:57.449928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:57.450586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:57.450870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:57.451150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:57.479388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:57.482287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:57.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:57.524329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:57.529824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:57.529824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:57.529824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:57.529824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 279/557] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 476 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:57.542077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:57.604034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:57.632695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfb54c-e976-47cc-856c-cfafd5080032 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:57.672880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:57.676690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:57.677360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:57.677677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:57.677956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:57.692702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:57.694285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:57.714608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:57.716161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:57.717286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:57.717286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:57.717286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:57.721733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 279/558] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:57.722925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:57.750609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f8b45-6306-4f0a-89b3-35902e018e77 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:57.811945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:57.813099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:57.813676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:57.814235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:57.815462 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:57.816016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:57.818790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:57.842607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 280/559] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:16:57.843519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:57.904257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40a781-230c-40a7-a54e-c4f193b02968 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:57.949110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:57.955951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219db39-9f8d-4395-8232-4f2d490a7887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:57.956805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219db39-9f8d-4395-8232-4f2d490a7887 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:57.957187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219db39-9f8d-4395-8232-4f2d490a7887 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:57.957403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219db39-9f8d-4395-8232-4f2d490a7887 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:57.969815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219db39-9f8d-4395-8232-4f2d490a7887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5219db39-9f8d-4395-8232-4f2d490a7887 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:57.975357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:57.977286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 280/560] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:57.979019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219db39-9f8d-4395-8232-4f2d490a7887 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:57.998056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211b405-6780-4d95-b59f-7c17319c52f3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:58.031238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:58.045832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52633ba-a5e7-4478-af32-41270cf99c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:58.046460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52633ba-a5e7-4478-af32-41270cf99c4b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.046744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52633ba-a5e7-4478-af32-41270cf99c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.047986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52633ba-a5e7-4478-af32-41270cf99c4b None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.048865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52633ba-a5e7-4478-af32-41270cf99c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:58.049179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52633ba-a5e7-4478-af32-41270cf99c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:58.050646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52633ba-a5e7-4478-af32-41270cf99c4b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:58.068492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 281/561] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:58.071223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047530c-e192-4c07-bde6-aaa69a876a8e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.071830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047530c-e192-4c07-bde6-aaa69a876a8e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.072417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047530c-e192-4c07-bde6-aaa69a876a8e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.075317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 282/562] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:58.076855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047530c-e192-4c07-bde6-aaa69a876a8e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:58.148011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219db39-9f8d-4395-8232-4f2d490a7887 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:58.194269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:58.195787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.196260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.196807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.210216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:58.213019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:16:58.235484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:58.236497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:58.237387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:16:58.237387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:16:58.237387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:16:58.237387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:16:58.237387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:16:58.237387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:16:58.237387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:16:58.237387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:58.238003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 281/563] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:16:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:58.239556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:58.261849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047530c-e192-4c07-bde6-aaa69a876a8e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:58.305162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:58.316299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075ddf7-f905-49da-ad92-cddf372662c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:58.316991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075ddf7-f905-49da-ad92-cddf372662c7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.317272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075ddf7-f905-49da-ad92-cddf372662c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.317596 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075ddf7-f905-49da-ad92-cddf372662c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.318273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075ddf7-f905-49da-ad92-cddf372662c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:58.318576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075ddf7-f905-49da-ad92-cddf372662c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:58.319964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075ddf7-f905-49da-ad92-cddf372662c7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:58.330390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 283/564] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:16:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:58.334112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4556-4ebe-47b2-ac38-f5ec0b46c856 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.334602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4556-4ebe-47b2-ac38-f5ec0b46c856 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.335083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c4556-4ebe-47b2-ac38-f5ec0b46c856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.361367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-090c4556-4ebe-47b2-ac38-f5ec0b46c856 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:58.362659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-090c4556-4ebe-47b2-ac38-f5ec0b46c856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:58.371225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c4ba2-f66e-4a1d-b997-cf2a33148bf0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:58.418564 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:58.419340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.419689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.420034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.425724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:58.431548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:58.433302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:16:58.448416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:58.449524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 282/565] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:16:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:58.450475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:58.467517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 284/566] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:58.469279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b72301-fbc9-483f-91c6-be0a40503d12 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.469854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b72301-fbc9-483f-91c6-be0a40503d12 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.470309 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b72301-fbc9-483f-91c6-be0a40503d12 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.472112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 285/567] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:16:58.473248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b72301-fbc9-483f-91c6-be0a40503d12 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:58.614287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63107c-a422-488f-9a95-cb1367a25e31 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:58.630497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:58.661549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:58.662181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.662464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.662738 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.663440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:58.663756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:58.664952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b72301-fbc9-483f-91c6-be0a40503d12 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:58.665228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:58.686409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 283/568] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:16:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:58.686409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:58.732714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890a300-3ca6-4cc5-8adc-84486273d176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:58.733698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890a300-3ca6-4cc5-8adc-84486273d176 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.734203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890a300-3ca6-4cc5-8adc-84486273d176 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.734701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890a300-3ca6-4cc5-8adc-84486273d176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.753358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890a300-3ca6-4cc5-8adc-84486273d176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:58.756466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890a300-3ca6-4cc5-8adc-84486273d176 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4890a300-3ca6-4cc5-8adc-84486273d176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:58.783865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:58.785718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:58.787246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:58.787246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:58.787246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:58.787246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 286/569] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:58.787246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8fc71-4c7f-4d39-989e-c6a037765b81 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.787246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8fc71-4c7f-4d39-989e-c6a037765b81 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.787246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8fc71-4c7f-4d39-989e-c6a037765b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.807822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-02f8fc71-4c7f-4d39-989e-c6a037765b81 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:16:58.809049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-02f8fc71-4c7f-4d39-989e-c6a037765b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:16:58.837647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:58.839037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24644f92-d014-4d51-8f97-00b88b4bcc27 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:58.893650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:58.894595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:58.895040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:58.895487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:58.896791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:58.897300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:58.899643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:58.904613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 287/570] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:16:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:16:58.905920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8fc71-4c7f-4d39-989e-c6a037765b81 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:58.931534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 284/571] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:16:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:58.931534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:58.974584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:59.057307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8fc71-4c7f-4d39-989e-c6a037765b81 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:59.099691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:59.100599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:59.101037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:59.101687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:59.103018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=4ba4c42b47794c9ba5fc6ee557e2cfc3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:59.103512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:59.105522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:59.112841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be1d36-a2d3-48ab-8e6e-c7b8a1bad16a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:59.113541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 288/572] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:16:58 2022] GET /identity/v3/role_assignments?user.id=4ba4c42b47794c9ba5fc6ee557e2cfc3&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:59.114547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:59.160917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:59.180482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:59.181524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:59.182010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:59.182490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:59.183691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:59.184158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:59.186696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:59.205394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 285/573] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:59.206618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:59.279313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e49e96-cafd-4e61-87bf-86f6b48e5dea None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:59.316774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:59.323589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bba8be-0c5c-469f-bea2-0d2b443c6723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:59.324261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba8be-0c5c-469f-bea2-0d2b443c6723 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:59.324546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba8be-0c5c-469f-bea2-0d2b443c6723 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:59.324824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba8be-0c5c-469f-bea2-0d2b443c6723 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:59.330807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40799f62-aa49-415e-8b2e-ab86a4545e2d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:59.340951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bba8be-0c5c-469f-bea2-0d2b443c6723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-76bba8be-0c5c-469f-bea2-0d2b443c6723 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:59.351136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:59.352165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:59.353042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 289/574] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:59.353786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bba8be-0c5c-469f-bea2-0d2b443c6723 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:59.388066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:59.388748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:59.389033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:59.389310 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:59.407879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:59.409477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:16:59.426554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:59.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:16:59.429133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:16:59.429133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:16:59.429133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:16:59.429133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 286/575] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:59.429133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:59.511496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:59.538621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bba8be-0c5c-469f-bea2-0d2b443c6723 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:59.561142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae33b8-d933-4ca0-9dd3-75ddbb0967df None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:59.582158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:59.582722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:59.583190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:59.583468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:59.592773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:59.594932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:16:59.607881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:59.608945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 290/576] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:16:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:59.611198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:59.622095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:59.623100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:59.623659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:59.624143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:59.625638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:59.626176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:59.627821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:59.638259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 287/577] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:16:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:59.639385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:59.663822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:59.753205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc4040-ea1e-4f0d-91e4-3ea79063f067 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:59.798013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d49223-12be-4dee-a233-e57d1717ec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:59.799096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d49223-12be-4dee-a233-e57d1717ec9a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:59.799587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d49223-12be-4dee-a233-e57d1717ec9a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:59.800068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d49223-12be-4dee-a233-e57d1717ec9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:59.816259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d49223-12be-4dee-a233-e57d1717ec9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-27d49223-12be-4dee-a233-e57d1717ec9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:16:59.822345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:16:59.824415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:16:59.826098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:16:59.829713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 291/578] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:16:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:16:59.830230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d49223-12be-4dee-a233-e57d1717ec9a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:16:59.833716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:16:59.834142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ea3c6-016a-41b6-9fd8-907ac1e01345 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:16:59.899939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:16:59.900990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:16:59.901339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:16:59.902296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:16:59.902614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:16:59.903155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:16:59.905322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:16:59.938552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 288/579] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:16:59.938552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:00.013225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d49223-12be-4dee-a233-e57d1717ec9a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:00.050692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:00.079772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:00.080842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:00.081316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:00.081834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:00.083157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:00.083662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:00.086496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:00.111894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 292/580] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Oct 3 13:16:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:00.112936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:00.123482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6223a45-e916-452b-ae4d-8d97d94a712e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:00.188283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:00.189606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:00.190102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:00.190548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:00.209102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:00.211804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:00.235932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:00.239044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:00.240017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:00.241038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:00.241038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:00.241038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:00.241213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 289/581] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:16:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:00.242673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:00.305149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cde92-58aa-4c4f-8456-0fb98a5b4e55 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:00.358427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:00.359183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:00.359525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:00.360028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:00.360938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:00.361325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:00.362981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:00.373820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 293/582] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:00.374630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:00.447376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e375b97-1929-4162-afcd-2b96e6edf60f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:00.468006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:00.492254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:00.492897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:00.493179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:00.493488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:00.502199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:00.503762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:00.517130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:00.519129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 290/583] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:00.520993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:00.549206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979388c-8edc-4de9-a5d7-988ddcf11d6c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:00.587105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:00.611173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:00.612158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:00.612643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:00.613221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:00.633114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:00.635631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:00.658635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:00.660501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:00.662213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:00.662213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:00.662213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:00.662213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 294/584] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:00.662213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:00.713158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b6d42-1d18-49a6-b398-4762b2f6a09d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:00.757135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:00.757799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:00.758078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:00.758350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:00.759017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:00.759301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:00.760707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:00.773716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 291/585] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:00.774594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:00.808340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa329e-fad8-4826-9b94-7fd7b2bb9453 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:00.821569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:00.867287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:00.868218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:00.868714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:00.869180 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:00.888378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:00.891054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:00.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:00.917072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:00.917996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:00.917996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:00.917996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:00.917996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 295/586] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:00.919041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:00.944517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f9b2c-af72-468e-8da4-0eb1919b8153 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:00.962888 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:00.988876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:00.989766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:00.990184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:00.990651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:00.991582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:00.992106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:00.993866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.015607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 292/587] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:01.015607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:01.069792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb677efd-feb6-41c0-bfba-c6e151a55a65 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:01.111901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:01.112523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:01.112803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:01.113082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:01.113768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:01.114078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:01.115493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.134578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 296/588] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:01.134578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:01.146741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:01.153816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc46c3-069a-4060-97cf-6d7345b78db7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:01.215559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:01.216660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:01.217154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:01.217684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:01.231961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:01.234795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:01.258572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:01.259585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:01.260632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 293/589] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:01.262135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:01.271241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:01.298924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97782d41-8961-4d2b-9f2d-90978d112c64 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:01.362915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:01.363889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:01.364334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:01.364794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:01.366066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:01.366586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:01.369085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.388406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 297/590] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:01.389204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:01.400154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121a356-eb43-4e9b-a022-3cf1f947db48 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:01.448176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8047-80e7-4364-9d33-a51fce410fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:01.449310 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8047-80e7-4364-9d33-a51fce410fb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:01.449884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8047-80e7-4364-9d33-a51fce410fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:01.450327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c8047-80e7-4364-9d33-a51fce410fb2 None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c3ece3405a2b43288136ddb79b84a106/roles/c49c853797c4458d84ca51c515a08755` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:01.470954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c8047-80e7-4364-9d33-a51fce410fb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=c3ece3405a2b43288136ddb79b84a106, role_id=c49c853797c4458d84ca51c515a08755)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:01.472773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c8047-80e7-4364-9d33-a51fce410fb2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.491352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 294/591] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:01 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c3ece3405a2b43288136ddb79b84a106/roles/c49c853797c4458d84ca51c515a08755 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:17:01.492646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8047-80e7-4364-9d33-a51fce410fb2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:01.512312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce811694-7b7a-4fc5-bc87-32a2fc2462be None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:01.522263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:01.559489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:01.560330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:01.560710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:01.561065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:01.570868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:01.573236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:01.590150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:01.591133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:01.592031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:01.592031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:01.592031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:01.592031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:01.592031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:01.592031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:01.592031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:01.592031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:01.597620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 298/592] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:01.598808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:01.613035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:01.660470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c8047-80e7-4364-9d33-a51fce410fb2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:01.700045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:01.700839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:01.701229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:01.701674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:01.702534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:01.702950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:01.705623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.733056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 295/593] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Oct 3 13:17:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:01.734286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:01.806950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:01.815039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f447b7-b6aa-4e0f-943a-6fb7f9223d2c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:01.853478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab033db-3b67-42a4-b763-24dc7cc7422d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:01.899713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:01.900777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:01.901061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:01.901340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:01.909136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:01.910717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:01.911470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:01.912252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:01.912731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:01.914238 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:01.914428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.915358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:01.918623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:01.938376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:01.940096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 13:17:01.941991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:01.945825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 299/594] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:01.947320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:01.950990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 296/595] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:01.960298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:02.077739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:02.110850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56079ca7-a19c-41b3-9bb5-eb98dd23dace None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:02.155349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:02.156036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36fbad-de3c-4cb0-9d4b-8c229a6a3534 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:02.179669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466b3ef-3b4d-4063-834e-1696411a3d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:02.180760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466b3ef-3b4d-4063-834e-1696411a3d9f None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:02.181264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466b3ef-3b4d-4063-834e-1696411a3d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:02.181801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466b3ef-3b4d-4063-834e-1696411a3d9f None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/464190e0ccaf4f82942f518df1bf6c03/roles/c49c853797c4458d84ca51c515a08755` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:02.208472 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466b3ef-3b4d-4063-834e-1696411a3d9f None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=464190e0ccaf4f82942f518df1bf6c03, role_id=c49c853797c4458d84ca51c515a08755)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:02.211212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466b3ef-3b4d-4063-834e-1696411a3d9f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:02.231811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fa89e-1435-4de6-be1b-549b551336a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:02.233321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa89e-1435-4de6-be1b-549b551336a0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:02.233737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 297/596] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:01 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/464190e0ccaf4f82942f518df1bf6c03/roles/c49c853797c4458d84ca51c515a08755 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:17:02.234324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa89e-1435-4de6-be1b-549b551336a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:02.235693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fa89e-1435-4de6-be1b-549b551336a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:02.236059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466b3ef-3b4d-4063-834e-1696411a3d9f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:02.267306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fa89e-1435-4de6-be1b-549b551336a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:02.272289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fa89e-1435-4de6-be1b-549b551336a0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-736fa89e-1435-4de6-be1b-549b551336a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:02.300909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:02.303046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:02.305065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:02.305065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:02.305065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:02.308542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 300/597] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:02.311934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50814ddc-92c2-4af9-84c1-162691eb231f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:02.313530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50814ddc-92c2-4af9-84c1-162691eb231f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:02.314269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50814ddc-92c2-4af9-84c1-162691eb231f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:02.318599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 301/598] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:02.322807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50814ddc-92c2-4af9-84c1-162691eb231f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:02.486047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466b3ef-3b4d-4063-834e-1696411a3d9f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:02.522274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50814ddc-92c2-4af9-84c1-162691eb231f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:02.542245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:02.558799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:02.560372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:02.561406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:02.561616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02836e7a-0075-4cf2-942b-4da79a1905e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:02.562296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02836e7a-0075-4cf2-942b-4da79a1905e9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:02.562607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02836e7a-0075-4cf2-942b-4da79a1905e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:02.562911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02836e7a-0075-4cf2-942b-4da79a1905e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:02.563537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02836e7a-0075-4cf2-942b-4da79a1905e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:02.563842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02836e7a-0075-4cf2-942b-4da79a1905e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:02.564700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:02.565283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02836e7a-0075-4cf2-942b-4da79a1905e9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:02.566808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:02.567795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:02.570985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:02.580094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 302/599] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:02.581524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b368e-a1fe-4dc5-9e5a-2f931d5a0c9a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:02.581834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b368e-a1fe-4dc5-9e5a-2f931d5a0c9a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:02.582103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b368e-a1fe-4dc5-9e5a-2f931d5a0c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:02.602762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b98b368e-a1fe-4dc5-9e5a-2f931d5a0c9a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:02.604020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b98b368e-a1fe-4dc5-9e5a-2f931d5a0c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:02.615973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 298/600] 10.222.0.58 () {56 vars in 1139 bytes} [Mon Oct 3 13:17:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:02.618805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:02.683442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:02.725814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 303/601] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:02.726873 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b368e-a1fe-4dc5-9e5a-2f931d5a0c9a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:02.823904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7793230-1144-4c58-ada7-5ad358c1ccee None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:02.938723 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:02.940322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:02.940322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:02.940680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:02.955717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:02.957228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:02.962030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b368e-a1fe-4dc5-9e5a-2f931d5a0c9a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:02.983642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:02.986312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:02.989933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 299/602] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:02.992871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:02.996834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:03.040867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:03.041887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:03.042356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:03.042794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:03.056771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:03.059010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:03.077898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:03.079567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:03.080584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:03.080584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:03.080584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:03.082537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 304/603] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:03.090288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:03.233271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:03.245919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e11004-0c28-4859-94b3-27b4ace8224c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:03.276084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f26ed2-3258-4c37-a63d-1671c518acdf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:03.291184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:03.291820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:03.292102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:03.292378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:03.293094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:03.293407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:03.295378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:03.310097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 300/604] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:03.311027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:03.355178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:03.356714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:03.357683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:03.358442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:03.361399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:03.362318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:03.365694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:03.393014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 305/605] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:03.394636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:03.444387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:03.544828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d06983-19ea-4e7c-a5a5-803a86639159 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:03.607737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a9240-9292-44a7-a3fa-b28366e41ea9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:03.615121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8378f-e282-49bc-ace6-03f639c55886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:03.616505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8378f-e282-49bc-ace6-03f639c55886 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:03.617234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8378f-e282-49bc-ace6-03f639c55886 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:03.617957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8378f-e282-49bc-ace6-03f639c55886 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:03.639476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8378f-e282-49bc-ace6-03f639c55886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-a9e8378f-e282-49bc-ace6-03f639c55886 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:03.650594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:03.652640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:03.654451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:03.658727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 301/606] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:03.660682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8378f-e282-49bc-ace6-03f639c55886 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:03.691064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875c9b9-408a-4507-9caa-37d1d53e8a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:03.692013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:03.692564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c9b9-408a-4507-9caa-37d1d53e8a2e None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:03.693814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c9b9-408a-4507-9caa-37d1d53e8a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:03.695469 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875c9b9-408a-4507-9caa-37d1d53e8a2e None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/8fa402682fd24228a77801edf1bb4d59/roles/c49c853797c4458d84ca51c515a08755` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:03.724618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875c9b9-408a-4507-9caa-37d1d53e8a2e None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=8fa402682fd24228a77801edf1bb4d59, role_id=c49c853797c4458d84ca51c515a08755)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:03.727819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875c9b9-408a-4507-9caa-37d1d53e8a2e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:03.759416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 306/607] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:03 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/8fa402682fd24228a77801edf1bb4d59/roles/c49c853797c4458d84ca51c515a08755 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:17:03.761234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875c9b9-408a-4507-9caa-37d1d53e8a2e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:03.955393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8378f-e282-49bc-ace6-03f639c55886 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:04.008401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:04.091327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875c9b9-408a-4507-9caa-37d1d53e8a2e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:04.098102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:04.098962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:04.099137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:04.099319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:04.116368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:04.117903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:04.131071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:04.133027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:04.134943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 302/608] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 474 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:04.139531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:04.176237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:04.177783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:04.178609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:04.179313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:04.180952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:04.181853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:04.184886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:04.203726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 307/609] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:04.214837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:04.266672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:04.378312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8634c3-56f0-461d-b22f-615a85a48e20 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:04.396725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913c6a7-35b1-452b-a71c-6c18d6a0ac81 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:04.414296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:04.420770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:04.421920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:04.422589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:04.423197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:04.438298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:04.440352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:04.458162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:04.459185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:04.460088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:04.460088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:04.460088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:04.463281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 308/610] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:04.472311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:04.514318 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:04.515140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:04.516212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:04.516601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:04.517466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:04.517894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:04.519517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:04.555889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 303/611] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:04.564687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:04.603660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b75c6-932c-47bd-89a4-120742ec3eec None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:04.644163 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:04.645563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:04.646002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:04.646448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:04.647277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:04.647765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:04.649352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:04.663964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 309/612] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:04.675222 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:04.769897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:04.775336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5809257-77b4-4957-a2a4-45f7d3016929 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:04.850577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cbb40-ad24-4bc7-9595-5ac0b5bc0226 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:04.868244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:04.884526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86aaa6-65c3-4c19-89d4-e83a63239840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:04.885713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86aaa6-65c3-4c19-89d4-e83a63239840 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:04.886198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86aaa6-65c3-4c19-89d4-e83a63239840 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:04.886707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86aaa6-65c3-4c19-89d4-e83a63239840 None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/cf865cb3c7fc432884e4e308c397c83d/roles/c49c853797c4458d84ca51c515a08755` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:04.916824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86aaa6-65c3-4c19-89d4-e83a63239840 None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=cf865cb3c7fc432884e4e308c397c83d, role_id=c49c853797c4458d84ca51c515a08755)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:04.918937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86aaa6-65c3-4c19-89d4-e83a63239840 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:04.925590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceebdd2-9086-4de3-967e-fb5c516d971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:04.926653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceebdd2-9086-4de3-967e-fb5c516d971f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:04.927126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceebdd2-9086-4de3-967e-fb5c516d971f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:04.927592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceebdd2-9086-4de3-967e-fb5c516d971f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:04.940556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceebdd2-9086-4de3-967e-fb5c516d971f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:04.944321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceebdd2-9086-4de3-967e-fb5c516d971f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:04.948342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 304/613] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:04 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/cf865cb3c7fc432884e4e308c397c83d/roles/c49c853797c4458d84ca51c515a08755 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-eceebdd2-9086-4de3-967e-fb5c516d971f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:04.962368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:04.963444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:04.964439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:04.967186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 310/614] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:04.976849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86aaa6-65c3-4c19-89d4-e83a63239840 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:05.009238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37303d6d-1b1c-4e1c-9011-4b82488c8316 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.009926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37303d6d-1b1c-4e1c-9011-4b82488c8316 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.011311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37303d6d-1b1c-4e1c-9011-4b82488c8316 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.015282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 311/615] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:05.026284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084c64d-6764-44bf-b87c-496da68351ba None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.026757 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084c64d-6764-44bf-b87c-496da68351ba None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.027436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084c64d-6764-44bf-b87c-496da68351ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.080401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7084c64d-6764-44bf-b87c-496da68351ba None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:05.081690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7084c64d-6764-44bf-b87c-496da68351ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:05.129585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86aaa6-65c3-4c19-89d4-e83a63239840 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:05.183332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:05.184408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.184920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.185414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.186487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:05.187030 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:05.190097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:05.203817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 312/616] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:05.205652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92afe-dfa2-4e39-b426-4a8e31826ff4 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.205937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92afe-dfa2-4e39-b426-4a8e31826ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.206220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd92afe-dfa2-4e39-b426-4a8e31826ff4 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.212569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 313/617] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:05.214191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0917c-0d6f-4b1c-9550-aa4f8d46643d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.214459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0917c-0d6f-4b1c-9550-aa4f8d46643d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.214744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0917c-0d6f-4b1c-9550-aa4f8d46643d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.217137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 305/618] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Oct 3 13:17:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:05.226785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:05.242262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cec0917c-0d6f-4b1c-9550-aa4f8d46643d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:05.243388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cec0917c-0d6f-4b1c-9550-aa4f8d46643d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:05.280750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:05.285952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:05.341634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 314/619] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:05.371906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701211-9372-48ca-991b-be988da78b95 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:05.415776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:05.416818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.417307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.417825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.440764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:05.442694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:05.459752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:05.460975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:05.462008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:05.462008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:05.462008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:05.464190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 306/620] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:05.480030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0917c-0d6f-4b1c-9550-aa4f8d46643d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:05.639462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:05.646966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0917c-0d6f-4b1c-9550-aa4f8d46643d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:05.711972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50480786-6194-4326-bb57-4381c7554560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:05.713438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50480786-6194-4326-bb57-4381c7554560 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.714235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50480786-6194-4326-bb57-4381c7554560 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.714902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50480786-6194-4326-bb57-4381c7554560 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.716147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50480786-6194-4326-bb57-4381c7554560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:05.716880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50480786-6194-4326-bb57-4381c7554560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:05.719821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50480786-6194-4326-bb57-4381c7554560 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:05.748742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 315/621] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:05.754724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50480786-6194-4326-bb57-4381c7554560 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:05.803374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:05.888387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:05.905774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50480786-6194-4326-bb57-4381c7554560 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:05.913749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4465d-10a4-4201-b52f-132baa48c511 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:05.969877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:05.971286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.971286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.971594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.974668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577ce0c-4759-4929-83ce-e9b909183e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:05.976250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ce0c-4759-4929-83ce-e9b909183e11 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:05.976731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ce0c-4759-4929-83ce-e9b909183e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:05.977212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577ce0c-4759-4929-83ce-e9b909183e11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:05.980188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:05.981806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:05.995283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:05.997483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 316/622] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:05.999489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:06.000788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577ce0c-4759-4929-83ce-e9b909183e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-9577ce0c-4759-4929-83ce-e9b909183e11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:06.014878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:06.017528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:06.019523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 307/623] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:06.021070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577ce0c-4759-4929-83ce-e9b909183e11 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:06.170129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:06.193496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4b5dd-2c58-42ed-a6e2-29e839d33067 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:06.234932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa229e-af5c-49ab-abdf-76bf0fab3acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:06.235977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa229e-af5c-49ab-abdf-76bf0fab3acf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:06.236266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa229e-af5c-49ab-abdf-76bf0fab3acf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:06.236544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa229e-af5c-49ab-abdf-76bf0fab3acf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:06.246459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577ce0c-4759-4929-83ce-e9b909183e11 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:06.248861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa229e-af5c-49ab-abdf-76bf0fab3acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6dfa229e-af5c-49ab-abdf-76bf0fab3acf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:06.255705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:06.258160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 317/624] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:06.260672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa229e-af5c-49ab-abdf-76bf0fab3acf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:06.340399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:06.340811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:06.341098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:06.341368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:06.342443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:06.342609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:06.347370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:06.361314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 308/625] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:06.362259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:06.381917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:06.464749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa229e-af5c-49ab-abdf-76bf0fab3acf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:06.522514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb95042-cdd7-4b63-94a5-224b8102ad37 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:06.542555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:06.543701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:06.543701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:06.544010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:06.544558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:06.544872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:06.546318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:06.557500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 318/626] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:06.558359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:06.569552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:06.594841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:06.596417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:06.597241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:06.598022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:06.599570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:06.600423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:06.603326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:06.620114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 309/627] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:06.627395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:06.684199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13d3f8-7c96-4845-8e39-d1aa347eda0a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:06.732751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6c16a-8499-4da4-9e43-f8574c7db87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:06.733336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c16a-8499-4da4-9e43-f8574c7db87e None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:06.733652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c16a-8499-4da4-9e43-f8574c7db87e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:06.733939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c16a-8499-4da4-9e43-f8574c7db87e None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/4ba4c42b47794c9ba5fc6ee557e2cfc3/roles/c49c853797c4458d84ca51c515a08755` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:06.753700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6c16a-8499-4da4-9e43-f8574c7db87e None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=4ba4c42b47794c9ba5fc6ee557e2cfc3, role_id=c49c853797c4458d84ca51c515a08755)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:06.755235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6c16a-8499-4da4-9e43-f8574c7db87e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:06.773177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 319/628] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:06 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/4ba4c42b47794c9ba5fc6ee557e2cfc3/roles/c49c853797c4458d84ca51c515a08755 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:17:06.774081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6c16a-8499-4da4-9e43-f8574c7db87e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:06.783231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:06.827734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a19a8-86de-442e-b397-68a7b7a2eb21 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:06.871393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:06.902063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:06.903470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:06.904195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:06.904897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:06.933668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:06.937052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:06.960497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6c16a-8499-4da4-9e43-f8574c7db87e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:06.994174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:06.996031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:06.997873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:06.997873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:06.997873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:07.002169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 310/629] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:07.010902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:07.030663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:07.031817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:07.032287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:07.032572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:07.045161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:07.047260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:07.068237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:07.070130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:07.071844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:07.071844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:07.071844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:07.071844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 320/630] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:07.075850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:07.206765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21082fb5-22dc-411d-9919-6e3837887b81 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:07.217086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:07.245872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:07.246397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:07.246686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:07.246960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:07.247623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:07.247934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:07.249344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:07.263929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 321/631] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:07.272667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:07.336925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88465f80-98f4-42a6-87c3-f65c52842532 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:07.417887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:07.445399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:07.446429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:07.446907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:07.447382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:07.448534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:07.449048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:07.451573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:07.486328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 311/632] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:07.495351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:07.566452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6335a3-33e1-4e41-b183-bccffce5c694 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:07.646645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b821a8-af77-4266-ac78-a2cca65a53a1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:07.653544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b272e4-ec8f-48d0-8918-06e9532bef7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:07.654626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b272e4-ec8f-48d0-8918-06e9532bef7e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:07.655124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b272e4-ec8f-48d0-8918-06e9532bef7e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:07.655605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b272e4-ec8f-48d0-8918-06e9532bef7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:07.669401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b272e4-ec8f-48d0-8918-06e9532bef7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:07.670971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b272e4-ec8f-48d0-8918-06e9532bef7e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d2b272e4-ec8f-48d0-8918-06e9532bef7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:07.684426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:07.687200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 322/633] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:17:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2c53f-a0e3-4c3e-9d6c-b86d853b1175 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:07.689732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2c53f-a0e3-4c3e-9d6c-b86d853b1175 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:07.691434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2c53f-a0e3-4c3e-9d6c-b86d853b1175 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:07.691434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 323/634] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:07.699050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da9645-07c2-4b22-b927-5c75ef91dce4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:07.699465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da9645-07c2-4b22-b927-5c75ef91dce4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:07.699661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da9645-07c2-4b22-b927-5c75ef91dce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:07.711502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:07.712768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:07.713258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:07.713803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:07.721340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d5da9645-07c2-4b22-b927-5c75ef91dce4 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:07.722655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d5da9645-07c2-4b22-b927-5c75ef91dce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:07.730298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:07.733295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:07.750882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:07.763210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:07.765189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 03 13:17:07.767657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:07.777775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 312/635] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:07.780022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:07.804297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 324/636] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:07.805189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da9645-07c2-4b22-b927-5c75ef91dce4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:07.861235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:07.919443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da9645-07c2-4b22-b927-5c75ef91dce4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:07.971589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:07.973528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:07.973889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:07.975861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:07.976695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:07.978323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:07.979901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:08.009808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1896e5-b349-40fb-9f65-0ed6e7546e09 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:08.029229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 325/637] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Oct 3 13:17:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:08.037606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:08.063818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:08.100457 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fe386-e3a0-4b84-b8ab-410d90f15e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:08.102242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fe386-e3a0-4b84-b8ab-410d90f15e64 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:08.103148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fe386-e3a0-4b84-b8ab-410d90f15e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:08.104041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fe386-e3a0-4b84-b8ab-410d90f15e64 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:08.105904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fe386-e3a0-4b84-b8ab-410d90f15e64 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:08.107091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fe386-e3a0-4b84-b8ab-410d90f15e64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:08.110956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fe386-e3a0-4b84-b8ab-410d90f15e64 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:08.150393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 313/638] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:17:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:08.153585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d2c8c-102b-40f0-b256-29309b695bd5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:08.155266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d2c8c-102b-40f0-b256-29309b695bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:08.156974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d2c8c-102b-40f0-b256-29309b695bd5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:08.160840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 314/639] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:08.163008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d2c8c-102b-40f0-b256-29309b695bd5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:08.191542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ee4a5-f85e-41a3-a32a-9664befce0a6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:08.250142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f59969-2223-4530-97b0-9096c62f0c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:08.250967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f59969-2223-4530-97b0-9096c62f0c72 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:08.251380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f59969-2223-4530-97b0-9096c62f0c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:08.251785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f59969-2223-4530-97b0-9096c62f0c72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:08.347530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f59969-2223-4530-97b0-9096c62f0c72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:08.350389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f59969-2223-4530-97b0-9096c62f0c72 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:08.366963 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d2c8c-102b-40f0-b256-29309b695bd5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:08.372509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:08.381620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:08.442370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e00bd5-f638-40d6-9783-6cae2858c798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:08.443007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e00bd5-f638-40d6-9783-6cae2858c798 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:08.443302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e00bd5-f638-40d6-9783-6cae2858c798 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:08.443739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e00bd5-f638-40d6-9783-6cae2858c798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:08.520746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 326/640] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:17:08.520746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db9a6-a598-4b37-9ca4-90f41a921ae5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:08.520746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db9a6-a598-4b37-9ca4-90f41a921ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:08.520746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6db9a6-a598-4b37-9ca4-90f41a921ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:08.537395 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4a6db9a6-a598-4b37-9ca4-90f41a921ae5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:08.538778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4a6db9a6-a598-4b37-9ca4-90f41a921ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:08.571853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e00bd5-f638-40d6-9783-6cae2858c798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:08.574741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e00bd5-f638-40d6-9783-6cae2858c798 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:08.639370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 327/641] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:08.641563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6db9a6-a598-4b37-9ca4-90f41a921ae5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:08.742718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 315/642] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:17:08.744344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e00bd5-f638-40d6-9783-6cae2858c798 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:08.753900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:08.757757 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:08.834064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6db9a6-a598-4b37-9ca4-90f41a921ae5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:08.907523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacd68d-fb54-47c9-a251-31a59336faa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:08.908643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacd68d-fb54-47c9-a251-31a59336faa0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:08.909136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacd68d-fb54-47c9-a251-31a59336faa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:08.909655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacd68d-fb54-47c9-a251-31a59336faa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:08.922642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacd68d-fb54-47c9-a251-31a59336faa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-afacd68d-fb54-47c9-a251-31a59336faa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:08.928015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:08.929036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:08.929933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:08.938021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 328/643] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:08.939484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacd68d-fb54-47c9-a251-31a59336faa0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:08.955357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e00bd5-f638-40d6-9783-6cae2858c798 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:09.005581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:09.007043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:09.008106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:09.008677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:09.033027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:09.035910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:09.082505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:09.084400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:09.095052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:09.095052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:09.095052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:09.095052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 316/644] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:09.095052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:09.117774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacd68d-fb54-47c9-a251-31a59336faa0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:09.126763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:09.161343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:09.162196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:09.162393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:09.162674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:09.176768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:09.178614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:09.204474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:09.206367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:09.209443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:09.209443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:09.209443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:09.209443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 329/645] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:09.209443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:09.220982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:09.285711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b41c7-92fa-4d5c-acbc-8ea8c4fa7a6f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:09.351172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:09.352239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:09.352733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:09.353213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:09.354479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:09.355026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:09.357696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:09.375321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 317/646] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:09.376931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:09.391143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91ec5-10b2-4365-96c9-93d067dbccab None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:09.449650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:09.450698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:09.451225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:09.451701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:09.453512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:09.454066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:09.456883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:09.487474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 330/647] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:09.487474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:09.523956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:09.567462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4246f-ea70-4d64-b142-0760d1a72e52 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:09.593572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:09.626854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d79c9-747f-4c72-ab98-2eaff968c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:09.627960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d79c9-747f-4c72-ab98-2eaff968c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:09.628455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d79c9-747f-4c72-ab98-2eaff968c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:09.628951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d79c9-747f-4c72-ab98-2eaff968c7e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:09.649880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d79c9-747f-4c72-ab98-2eaff968c7e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:09.652784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72e9be-dfbc-451e-b5d1-ec99af70fdcd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-639d79c9-747f-4c72-ab98-2eaff968c7e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:09.661246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:09.663148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:09.664764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 318/648] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:09.666281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d79c9-747f-4c72-ab98-2eaff968c7e4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:09.704805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:09.705818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:09.706308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:09.706782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:09.708124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:09.708640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:09.710347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:09.725898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 331/649] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:09.727062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:09.823320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:09.829824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d79c9-747f-4c72-ab98-2eaff968c7e4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:09.846516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d6962-982a-4e76-9437-1e30a447e4a5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:09.871896 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:09.872971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:09.873436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:09.874082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:09.885129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:09.885831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:09.886173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:09.886707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:09.888436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:09.890781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:09.895873 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:09.897437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:09.911387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:09.913170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 332/650] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:09.915404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:09.916485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:09.919594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:09.922972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:09.922972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:09.922972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:09.922972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 319/651] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:09.922972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:09.957873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:10.057280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86af9d7-356b-4e68-9c9a-194fce6448d8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:10.065592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e57ab9-de8d-4e98-917c-456a200f68b2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:10.099504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:10.100139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.100525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.100870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.110383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:10.111482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ff3d1-7d19-4140-94d0-80564a7af92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:10.112108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff3d1-7d19-4140-94d0-80564a7af92f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.112224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:10.112389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff3d1-7d19-4140-94d0-80564a7af92f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.112667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ff3d1-7d19-4140-94d0-80564a7af92f None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.113357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ff3d1-7d19-4140-94d0-80564a7af92f None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:10.113697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ff3d1-7d19-4140-94d0-80564a7af92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:10.115118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ff3d1-7d19-4140-94d0-80564a7af92f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:10.120973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:10.127254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 333/652] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:10.127837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:10.129654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:10.131633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:10.131633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:10.131633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:10.131633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:10.131633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:10.131633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:10.131633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:10.131633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:10.132559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948902e-784f-4620-a500-ea6ba4ab7b3c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.132559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948902e-784f-4620-a500-ea6ba4ab7b3c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.132559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948902e-784f-4620-a500-ea6ba4ab7b3c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.133848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 334/654] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:10.134068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 320/654] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:10.135085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948902e-784f-4620-a500-ea6ba4ab7b3c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:10.136136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:10.252778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:10.266774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ac6ac-3404-44f9-8b28-7750477d141c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:10.290768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948902e-784f-4620-a500-ea6ba4ab7b3c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:10.307747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:10.308377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.308657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.308927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.309644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:10.309949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:10.311365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:10.326437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 321/655] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:10.327304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:10.355343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587402d1-17d2-4edc-b602-96cf66283dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:10.356205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587402d1-17d2-4edc-b602-96cf66283dfd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.356543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587402d1-17d2-4edc-b602-96cf66283dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.357038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587402d1-17d2-4edc-b602-96cf66283dfd None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.358303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587402d1-17d2-4edc-b602-96cf66283dfd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:10.358814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587402d1-17d2-4edc-b602-96cf66283dfd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:10.361580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587402d1-17d2-4edc-b602-96cf66283dfd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:10.381798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 335/656] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:10.382352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2180ef-39ea-46d5-8676-60a706385b3a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.382698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2180ef-39ea-46d5-8676-60a706385b3a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.383032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2180ef-39ea-46d5-8676-60a706385b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.404994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8e2180ef-39ea-46d5-8676-60a706385b3a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:10.406583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8e2180ef-39ea-46d5-8676-60a706385b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:10.471325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cc655-8881-4b94-8b58-d6a97c43b1e8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:10.472688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:10.490066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 336/657] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:10.490952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2180ef-39ea-46d5-8676-60a706385b3a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:10.516382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:10.517000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.517286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.517597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.530444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:10.532527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:10.552934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:10.554162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:10.555179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:10.555179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:10.555179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:10.555179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 322/658] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:10.556136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:10.560059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:10.640887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2180ef-39ea-46d5-8676-60a706385b3a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:10.698723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:10.699755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.700268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.701550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.702892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:10.703445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:10.707651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:10.722952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 337/659] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:10.724185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:10.753500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b555b9-12f1-42a2-a89d-10773d290c8c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:10.803672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:10.804443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.804871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.805303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.816511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:10.818033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:10.821408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:10.831010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:10.842453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 323/660] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:10.845514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:10.912466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c47f03-fa12-40e1-bd35-fb3d6b3079fc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:10.927097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:10.954396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:10.955361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:10.955759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:10.956219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:10.957514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=464190e0ccaf4f82942f518df1bf6c03) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:10.958033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:10.960635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:10.983192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 338/661] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:10 2022] GET /identity/v3/role_assignments?user.id=464190e0ccaf4f82942f518df1bf6c03&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:10.983192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:10.983577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbeb75-c43b-471f-8ad2-037394eb1a9c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:11.030080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:11.031795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:11.032429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:11.032886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:11.033696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:11.034133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:11.035742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:11.051412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 324/662] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:11.052302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:11.174980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d298f6d-93ac-467e-9a7f-02f0d6d45f42 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:11.196825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:11.229606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6fb28-c3f2-455a-862e-7402f59727ac None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:11.232407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:11.232561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:11.232918 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:11.233733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:11.235908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c3ece3405a2b43288136ddb79b84a106) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:11.236055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:11.239148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:11.248839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 339/663] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:10 2022] GET /identity/v3/role_assignments?user.id=c3ece3405a2b43288136ddb79b84a106&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:11.251253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:11.278794 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:11.313997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73096451-7473-44f8-9f65-dbd122a880bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:11.314271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73096451-7473-44f8-9f65-dbd122a880bb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:11.314815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73096451-7473-44f8-9f65-dbd122a880bb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:11.315117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73096451-7473-44f8-9f65-dbd122a880bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:11.340879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73096451-7473-44f8-9f65-dbd122a880bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-73096451-7473-44f8-9f65-dbd122a880bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:11.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:11.353175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 325/664] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:11.360270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73096451-7473-44f8-9f65-dbd122a880bb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:11.479171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412c4b0-6f56-4664-8dd7-9ec99ea84f43 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:11.543028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73096451-7473-44f8-9f65-dbd122a880bb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:11.558990 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:11.561607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:11.563633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:11.564444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:11.566206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:11.567063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:11.570120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:11.585220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:11.585908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:11.586194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:11.586483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:11.596702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:11.598258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:11.606816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 340/665] 10.222.0.58 () {56 vars in 1137 bytes} [Mon Oct 3 13:17:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:11.608389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:11.616772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:11.618663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 326/666] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:11.621851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:11.676239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:11.690650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:11.748523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c72554-ab53-4b5a-87d3-df35b2e48bb0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:11.793298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f158c-fdd4-45f8-91e6-5c5575ebb92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:11.793986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f158c-fdd4-45f8-91e6-5c5575ebb92a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:11.794271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f158c-fdd4-45f8-91e6-5c5575ebb92a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:11.794671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f158c-fdd4-45f8-91e6-5c5575ebb92a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:11.835297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd21b3-5061-42a1-ba80-1b0d2503e680 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:11.886304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f158c-fdd4-45f8-91e6-5c5575ebb92a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:11.887894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f158c-fdd4-45f8-91e6-5c5575ebb92a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:11.915479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:11.916592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:11.917086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:11.917615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:11.919110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:11.919798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:11.923399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:11.941487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 341/667] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:11.942044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:11.944838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:12.019113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 327/668] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:17:12.019113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f158c-fdd4-45f8-91e6-5c5575ebb92a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:12.140772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f158c-fdd4-45f8-91e6-5c5575ebb92a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:12.158652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:12.162939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312df5-8053-4916-841d-a9037a0e9247 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:12.217267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:12.219911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:12.220762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:12.221940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:12.231541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:12.232643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:12.233140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:12.233662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:12.235897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:12.236520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:12.239345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:12.250808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:12.254127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:12.258853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:12.289079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 342/669] 10.222.0.58 () {56 vars in 1139 bytes} [Mon Oct 3 13:17:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:12.289079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:12.298071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:12.299863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:12.301544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:12.301544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:12.301544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:12.305678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 328/670] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:12.307288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:12.528791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de32ac1-9c47-46ff-87c6-afb24460d20b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:12.539657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde7889-ab0d-48d1-a09e-cdaaa09a922d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:12.597505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:12.599196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:12.600115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:12.600850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:12.608429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2417d-5ad9-4942-9884-a7e550456175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:12.609852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2417d-5ad9-4942-9884-a7e550456175 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:12.610433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2417d-5ad9-4942-9884-a7e550456175 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:12.611004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2417d-5ad9-4942-9884-a7e550456175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:12.623233 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:12.625893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:12.644601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:12.645705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:12.646609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:12.646609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:12.646609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:12.648984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 343/671] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:12.650243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:12.663299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:12.741559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:12.759764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2417d-5ad9-4942-9884-a7e550456175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:12.763218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2417d-5ad9-4942-9884-a7e550456175 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:12.768855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197e06d-d25a-4fb2-8243-38603cdb56b1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:12.807155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:12.808133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:12.808624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:12.809105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:12.810023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:12.810542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:12.812785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:12.832837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 344/672] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:12.834141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:12.911664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:12.952016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ac86f-157c-4ced-b84d-33b026531e19 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:12.993423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 329/673] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:17:12.993423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2417d-5ad9-4942-9884-a7e550456175 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:12.997841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:12.998774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:12.999315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.000392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.001342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:13.001887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:13.003506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:13.017804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 345/674] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:13.019030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:13.138246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dafdb-10c9-48e2-8e23-dabdc042aa01 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:13.169327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:13.179326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:13.180382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:13.180872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.181354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.194807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:13.195294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:13.197324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:13.209817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2417d-5ad9-4942-9884-a7e550456175 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:13.215235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:13.216213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:13.217133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:13.217133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:13.217133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:13.223906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 346/675] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:13.225294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:13.268196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:13.269851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:13.270523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.271062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.286158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:13.288954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:13.315984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:13.317040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:13.318049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:13.321768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 330/676] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:13.323433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:13.345378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b034-3ba0-4c3d-a5c8-07b5d35cf45b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:13.386714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:13.387837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:13.388437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.388924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.397370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:13.399228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:13.412859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:13.413879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:13.414794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:13.414794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:13.414794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:13.414794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:13.414794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:13.414794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:13.414794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:13.414794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:13.417385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 347/677] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:13.418523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:13.472807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:13.544253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde6191-5f12-4d1a-9756-3a13ad6229c1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:13.545883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0d792-1867-4da1-9026-592ee16cbc53 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:13.589088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:13.590259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:13.590784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.591279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.592277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:13.592821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:13.594853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:13.616300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:13.617425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:13.618055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.618521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 348/678] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Oct 3 13:17:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:13.619647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:13.624762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.626789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:13.627442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:13.630215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:13.665271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 331/679] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:13.666249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:13.698104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:13.742393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:13.746978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780f0d6-4d22-458e-b98d-494da137215f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:13.793182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:13.794229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:13.794722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.795207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.796120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:13.797565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:13.799341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:13.813842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 349/680] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:13.815139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:13.870679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e687a-f5c9-4b62-a793-30e9da950e4b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:13.941493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea56bf-eb5a-4e2f-9b80-08cb85f1df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:13.942636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea56bf-eb5a-4e2f-9b80-08cb85f1df44 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:13.943283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea56bf-eb5a-4e2f-9b80-08cb85f1df44 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.943695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea56bf-eb5a-4e2f-9b80-08cb85f1df44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.947994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0337075-90e4-448e-8ae6-2abee2384c1b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:13.988714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:13.989846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:13.989846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:13.990102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:13.998761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:14.000794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:14.013753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:14.014762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:14.015748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 350/681] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:14.016568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:14.050351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:14.056857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea56bf-eb5a-4e2f-9b80-08cb85f1df44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:14.058488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea56bf-eb5a-4e2f-9b80-08cb85f1df44 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:14.075421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:14.144759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1e092-6bd5-438e-9ee9-a810370c556a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:14.178376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 332/682] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:17:14.180041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59d9c0-a48b-40e5-9174-fc7db8dec982 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:14.180342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59d9c0-a48b-40e5-9174-fc7db8dec982 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:14.180620 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59d9c0-a48b-40e5-9174-fc7db8dec982 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:14.182076 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 333/683] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:14.183010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59d9c0-a48b-40e5-9174-fc7db8dec982 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:14.189978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73b4cf-7604-49fc-9dbf-a4091443ae1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:14.190631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73b4cf-7604-49fc-9dbf-a4091443ae1a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:14.190924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73b4cf-7604-49fc-9dbf-a4091443ae1a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:14.191445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73b4cf-7604-49fc-9dbf-a4091443ae1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:14.192318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73b4cf-7604-49fc-9dbf-a4091443ae1a None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=8fa402682fd24228a77801edf1bb4d59) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:14.192628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73b4cf-7604-49fc-9dbf-a4091443ae1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:14.194312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73b4cf-7604-49fc-9dbf-a4091443ae1a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:14.205545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 351/684] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:14 2022] GET /identity/v3/role_assignments?user.id=8fa402682fd24228a77801edf1bb4d59&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:14.205545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120e4ff-8757-4d71-8ad2-35b53f24240c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:14.205545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120e4ff-8757-4d71-8ad2-35b53f24240c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:14.206052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120e4ff-8757-4d71-8ad2-35b53f24240c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:14.215071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 352/685] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:14.215071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac85379-bfa6-4b47-b88c-c281314a8ec2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:14.215071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac85379-bfa6-4b47-b88c-c281314a8ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:14.215071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac85379-bfa6-4b47-b88c-c281314a8ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:14.243537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8ac85379-bfa6-4b47-b88c-c281314a8ec2 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:14.245592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8ac85379-bfa6-4b47-b88c-c281314a8ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:14.329424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59d9c0-a48b-40e5-9174-fc7db8dec982 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:14.354322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:14.369498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 353/686] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:14.369498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac85379-bfa6-4b47-b88c-c281314a8ec2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:14.383353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:14.394880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cefde-0d8c-4b76-91a3-240febd3a2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:14.395940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cefde-0d8c-4b76-91a3-240febd3a2eb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:14.396508 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cefde-0d8c-4b76-91a3-240febd3a2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:14.397320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cefde-0d8c-4b76-91a3-240febd3a2eb None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:14.398943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cefde-0d8c-4b76-91a3-240febd3a2eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:14.399463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cefde-0d8c-4b76-91a3-240febd3a2eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:14.402705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cefde-0d8c-4b76-91a3-240febd3a2eb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:14.429765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 334/687] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:14.433142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4713a-e25d-411c-a446-51474302caa5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:14.433657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4713a-e25d-411c-a446-51474302caa5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:14.434140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4713a-e25d-411c-a446-51474302caa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:14.470796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cea4713a-e25d-411c-a446-51474302caa5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:14.472778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cea4713a-e25d-411c-a446-51474302caa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:14.512256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac85379-bfa6-4b47-b88c-c281314a8ec2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:14.587226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:14.588797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:14.589298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:14.589867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:14.615508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:14.619409 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:14.639486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 335/688] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:14.642876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4713a-e25d-411c-a446-51474302caa5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:14.657801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:14.659845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:14.662224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:14.662224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:14.662224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:14.662224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 354/689] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:14.673760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:14.869063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:14.932438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4713a-e25d-411c-a446-51474302caa5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:14.938339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:14.969851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855fe64-4b05-411a-99ff-b035101106b6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:15.000376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.001413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.001949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.002433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.003811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=cf865cb3c7fc432884e4e308c397c83d) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:15.004374 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:15.007470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:15.021042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 336/690] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:14 2022] GET /identity/v3/role_assignments?user.id=cf865cb3c7fc432884e4e308c397c83d&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 608 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:15.023286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:15.050150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.052721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.053770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.054450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.055889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:15.057013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:15.061258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:15.101829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 355/691] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:15.103425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:15.237749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbd2d2-009c-4fcc-97bf-e3dbfaf82d35 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:15.287837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2ab76-8e0a-4f50-a9a5-0c633e6eb7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.288870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ab76-8e0a-4f50-a9a5-0c633e6eb7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.289159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ab76-8e0a-4f50-a9a5-0c633e6eb7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.289441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2ab76-8e0a-4f50-a9a5-0c633e6eb7a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.291813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7ff57-3c41-4571-971a-418f8b8d7219 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:15.305249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2ab76-8e0a-4f50-a9a5-0c633e6eb7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-7fa2ab76-8e0a-4f50-a9a5-0c633e6eb7a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:15.313615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:15.314675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:15.315562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:15.316229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 337/692] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:15.321581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2ab76-8e0a-4f50-a9a5-0c633e6eb7a4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:15.325123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:15.354857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.356006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.356512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.356981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.370594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:15.377648 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:15.381399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:15.410646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:15.411958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:15.413091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:15.413091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:15.413091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:15.413271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 356/693] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:15.414925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:15.459618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2ab76-8e0a-4f50-a9a5-0c633e6eb7a4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:15.501562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.502501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.502974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.503425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.511427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:15.513212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:15.525591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:15.526541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:15.527441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:15.527441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:15.527441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:15.527441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:15.527441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:15.527441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:15.527441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:15.527441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:15.534939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 338/694] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:15.536164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:15.626251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984e0fa-9552-4229-b3cd-40eafa02e64e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:15.636478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:15.651764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa3993-e50c-4714-b918-df1a69dc3e67 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:15.692459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.693519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.693983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.694431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.695289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:15.695769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:15.697568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:15.698804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.699889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.700402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.700905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.702290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:15.704037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:15.706690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:15.714158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 339/695] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:15.715462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:15.723593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 357/696] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:15.724976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:15.784015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:15.863012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511d0a8-0c6a-4650-9ae1-8d71fbfe6c11 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:15.881972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29852a-a4d2-4467-afc2-167274d42cd4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:15.908803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6548dd6-4431-4934-be37-9321c9028c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.909449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6548dd6-4431-4934-be37-9321c9028c7e None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.909912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6548dd6-4431-4934-be37-9321c9028c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.910197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6548dd6-4431-4934-be37-9321c9028c7e None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.910901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6548dd6-4431-4934-be37-9321c9028c7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:15.912444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6548dd6-4431-4934-be37-9321c9028c7e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:15.940573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 358/697] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:17:15 2022] POST /identity/v3/users => generated 254 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:15.941583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6548dd6-4431-4934-be37-9321c9028c7e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:15.959909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:15.961486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:15.962301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:15.963058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:15.964658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:15.965517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:15.969481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:15.990377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 340/698] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:16.003290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:16.035373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:16.063329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:16.073886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6548dd6-4431-4934-be37-9321c9028c7e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:16.131099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:16.132121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:16.136457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:16.136973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:16.159127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:16.161766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:16.184419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:16.186547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:16.188712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:16.188712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:16.188712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:16.190818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 359/699] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:16.196514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:16.273394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c04ef-87e2-4f1d-a947-b4423a4a74a5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:16.329066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946cc75-9103-452f-a662-e1397aab88c7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:16.359595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:16.360585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:16.361062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:16.361612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:16.363002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=4ba4c42b47794c9ba5fc6ee557e2cfc3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:16.363633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:16.367987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:16.370341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:16.370978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:16.371268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:16.371548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:16.372216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:16.372534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:16.373955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:16.381350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 341/700] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:16 2022] GET /identity/v3/role_assignments?user.id=4ba4c42b47794c9ba5fc6ee557e2cfc3&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 608 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:16.393269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 360/701] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:16.402339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:16.559780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:16.618937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c5d19-ad66-4b24-9a22-9652e1748174 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:16.690967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e9f38-09d7-4f02-8630-3f455b9a3542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:16.693579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e9f38-09d7-4f02-8630-3f455b9a3542 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:16.694481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e9f38-09d7-4f02-8630-3f455b9a3542 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:16.695273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e9f38-09d7-4f02-8630-3f455b9a3542 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:16.711525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e9f38-09d7-4f02-8630-3f455b9a3542 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:16.715024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e9f38-09d7-4f02-8630-3f455b9a3542 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-395e9f38-09d7-4f02-8630-3f455b9a3542 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:16.739502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:16.741339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:17:16.743178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:16.748212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 342/702] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:17:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:16.755197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:16.772138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:16.833502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9c6df-fc6f-4158-9523-469847a7284b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:16.834338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9c6df-fc6f-4158-9523-469847a7284b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:16.835001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9c6df-fc6f-4158-9523-469847a7284b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:16.837780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 343/703] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:16.847858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5500ec-ae49-4844-a595-4618a8ffb8ea None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:16.848346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5500ec-ae49-4844-a595-4618a8ffb8ea None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:16.848905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5500ec-ae49-4844-a595-4618a8ffb8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:16.884984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6a5500ec-ae49-4844-a595-4618a8ffb8ea None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:16.888513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6a5500ec-ae49-4844-a595-4618a8ffb8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:16.939225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c59e1-53ca-498b-be83-a9a59222e25c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:16.980074 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:16.980703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:16.980994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:16.981185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 344/704] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:16.981289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:16.982082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:16.982398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:16.984504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:17.006759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 361/705] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:17:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:17.012528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5500ec-ae49-4844-a595-4618a8ffb8ea None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:17.083152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:17.136094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5500ec-ae49-4844-a595-4618a8ffb8ea None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:17.186550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917ed17-19c7-4b8e-9601-103270ec7d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:17.188350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ed17-19c7-4b8e-9601-103270ec7d60 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:17.188910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ed17-19c7-4b8e-9601-103270ec7d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:17.189368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917ed17-19c7-4b8e-9601-103270ec7d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:17.289096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917ed17-19c7-4b8e-9601-103270ec7d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:17.290909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917ed17-19c7-4b8e-9601-103270ec7d60 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:17.399214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:17.443098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 345/706] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:17:17.448728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:17.539320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917ed17-19c7-4b8e-9601-103270ec7d60 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:17.541182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:17.587742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663e8d-e699-4bff-9069-c42765e39bcd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:17.627148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:17.627803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:17.628085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:17.628369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:17.640787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:17.642798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:17.660376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:17.661404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:17.662790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:17.662790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:17.662790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:17.662790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 362/707] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:17.668362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:17.792352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63ba46-cb93-4c40-aef3-d47e8bfca238 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:17.815840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:17.818446 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917ed17-19c7-4b8e-9601-103270ec7d60 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:17.845017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:17.847094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:17.847383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:17.847644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:17.848379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:17.848696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:17.851433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:17.879433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 363/708] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:17.885414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:17.908303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:17.909677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:17.910201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:17.910748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:17.911720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:17.913419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:17.927021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:17.928171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:17.929020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:17.932090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 346/709] 10.222.0.58 () {56 vars in 1129 bytes} [Mon Oct 3 13:17:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:17.938409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:17.987226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:18.074122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d04fd1-269f-4b50-af77-3a48f8b2ceaf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:18.128758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:18.129608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:18.130064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:18.130538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:18.157210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:18.162125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:18.173801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d24cfb-0de1-4de3-a042-5de5d57f43b0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:18.206755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:18.207735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:18.208626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:18.208626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:18.208626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:18.208626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:18.208626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:18.208626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:18.208626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:18.208626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:18.214427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 364/710] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:18.220774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:18.272141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5255017-6a85-47af-8024-8fff1647dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:18.272826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5255017-6a85-47af-8024-8fff1647dc6e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:18.273107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5255017-6a85-47af-8024-8fff1647dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:18.273431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5255017-6a85-47af-8024-8fff1647dc6e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:18.274193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5255017-6a85-47af-8024-8fff1647dc6e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:18.274506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5255017-6a85-47af-8024-8fff1647dc6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:18.275939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5255017-6a85-47af-8024-8fff1647dc6e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:18.281952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 347/711] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Oct 3 13:17:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:18.283309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5e04e-99c3-46ce-987e-f7f12ec9081c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:18.283602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5e04e-99c3-46ce-987e-f7f12ec9081c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:18.283867 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5e04e-99c3-46ce-987e-f7f12ec9081c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:18.289724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 348/712] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:18.291095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d16cb-c81a-4998-8b7d-0e8b13202b25 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:18.291383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d16cb-c81a-4998-8b7d-0e8b13202b25 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:18.291662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d16cb-c81a-4998-8b7d-0e8b13202b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:18.314394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:18.319279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b65d16cb-c81a-4998-8b7d-0e8b13202b25 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:18.322952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b65d16cb-c81a-4998-8b7d-0e8b13202b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:18.395508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4916b4-65f9-43fb-9e96-79ca56366c8a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:18.440828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:18.441562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:18.441847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:18.442125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:18.442910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:18.443143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:18.444636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:18.464191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 365/713] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:18.465268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:18.467307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 349/714] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:18.469588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d16cb-c81a-4998-8b7d-0e8b13202b25 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:18.509496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:18.642053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:18.648625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91e23f-c859-4501-a4e0-a418bdff7b9f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:18.670924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d16cb-c81a-4998-8b7d-0e8b13202b25 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:18.738567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:18.740059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:18.740854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:18.741628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:18.743236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c3ece3405a2b43288136ddb79b84a106) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:18.744054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:18.747195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:18.747386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a7329-a73f-426b-be11-9acb489ae0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:18.748012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7329-a73f-426b-be11-9acb489ae0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:18.748302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7329-a73f-426b-be11-9acb489ae0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:18.748585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7329-a73f-426b-be11-9acb489ae0e1 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:18.750195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a7329-a73f-426b-be11-9acb489ae0e1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:18.750195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a7329-a73f-426b-be11-9acb489ae0e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:18.751531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a7329-a73f-426b-be11-9acb489ae0e1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:18.761777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 350/715] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:18 2022] GET /identity/v3/role_assignments?user.id=c3ece3405a2b43288136ddb79b84a106&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:18.770878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 366/716] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Oct 3 13:17:18 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:19.067621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:19.253584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6998-4b82-4a4c-a9d1-5026dab46157 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:19.254187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6998-4b82-4a4c-a9d1-5026dab46157 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:19.254728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd6998-4b82-4a4c-a9d1-5026dab46157 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:19.261808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 367/717] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:19.266048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:19.268344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631621-d660-40de-8451-d24a35f00443 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:19.268673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631621-d660-40de-8451-d24a35f00443 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:19.268965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631621-d660-40de-8451-d24a35f00443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:19.289858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7dfae-2fd6-4be8-9b8f-fe72a725c990 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:19.303103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-37631621-d660-40de-8451-d24a35f00443 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:19.304343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-37631621-d660-40de-8451-d24a35f00443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:19.364740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cef2c-2985-4b5b-ab0b-026738b82c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:19.367854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cef2c-2985-4b5b-ab0b-026738b82c92 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:19.368988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cef2c-2985-4b5b-ab0b-026738b82c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:19.371674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cef2c-2985-4b5b-ab0b-026738b82c92 None admin] PATH_INFO: `/v3/services/network` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:19.373095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cef2c-2985-4b5b-ab0b-026738b82c92 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:19.380479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cef2c-2985-4b5b-ab0b-026738b82c92 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-685cef2c-2985-4b5b-ab0b-026738b82c92 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:19.397609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:19.399561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:19.401219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:19.404439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 351/718] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:19 2022] GET /identity/v3/services/network => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:19.406098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679e51-1bcf-4e7a-b98e-32e91ac9ef27 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:19.406403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679e51-1bcf-4e7a-b98e-32e91ac9ef27 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:19.406682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679e51-1bcf-4e7a-b98e-32e91ac9ef27 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:19.412703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 352/719] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:19.413804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa679e51-1bcf-4e7a-b98e-32e91ac9ef27 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:19.499100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 368/720] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:19.503784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3a595-89d0-4157-b555-85b06e4d2810 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:19.504573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3a595-89d0-4157-b555-85b06e4d2810 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:19.505263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3a595-89d0-4157-b555-85b06e4d2810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:19.532815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ade3a595-89d0-4157-b555-85b06e4d2810 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:19.534457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ade3a595-89d0-4157-b555-85b06e4d2810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:19.568540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa679e51-1bcf-4e7a-b98e-32e91ac9ef27 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:19.640037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:19.641981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:19.642932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:19.643573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:19.644942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] RBAC: Adding query filter params (name=network) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:19.652068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:19.655027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:19.658980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 369/721] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:19.664974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 353/722] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:19.670715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3a595-89d0-4157-b555-85b06e4d2810 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:19.710824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:19.827812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3a595-89d0-4157-b555-85b06e4d2810 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:19.880890 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:19.881553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:19.881843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:19.882116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:19.882768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] RBAC: Adding query filter params (type=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:19.883090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:19.884479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:19.890622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 370/723] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:19.991223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:20.028915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:20.080340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:20.204248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd60e7d-205c-446d-9233-707ab741278b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:20.254375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5a290-e803-4226-8834-eafee7fefd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:20.256200 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5a290-e803-4226-8834-eafee7fefd08 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:20.257042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5a290-e803-4226-8834-eafee7fefd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:20.257918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5a290-e803-4226-8834-eafee7fefd08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:20.280890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5a290-e803-4226-8834-eafee7fefd08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:20.285282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-36d5a290-e803-4226-8834-eafee7fefd08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:20.294108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:20.296797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:20.298813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:20.302970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 354/724] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:20.310520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5a290-e803-4226-8834-eafee7fefd08 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:20.349486 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf34b85-98c2-4639-aeeb-53f870a06d4c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:20.422696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0314f-e71e-410e-96a3-0e5e9115e0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:20.423351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0314f-e71e-410e-96a3-0e5e9115e0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:20.423638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0314f-e71e-410e-96a3-0e5e9115e0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:20.423919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0314f-e71e-410e-96a3-0e5e9115e0b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:20.445682 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0314f-e71e-410e-96a3-0e5e9115e0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-82e0314f-e71e-410e-96a3-0e5e9115e0b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:20.451099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:20.453600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:20.455374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:20.457519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 371/725] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:20.466681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0314f-e71e-410e-96a3-0e5e9115e0b1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:20.474200 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:20.548839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5a290-e803-4226-8834-eafee7fefd08 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:20.627973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:20.628689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:20.628981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:20.629257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:20.629966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:20.630272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:20.631947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:20.659110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 355/726] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:20.672482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:20.820343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0314f-e71e-410e-96a3-0e5e9115e0b1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:20.877831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:20.878570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:20.878856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:20.879133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:20.879825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:20.880130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:20.881822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:20.892222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696bf93-3df2-4c4d-8527-cd7ab26d249f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:20.892591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 372/727] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:20.901949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:20.914094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:20.970443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:20.971107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:20.971390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:20.971680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:20.974909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:20.992294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:20.994263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:21.044559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:21.045633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:21.046610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:21.046610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:21.046610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:21.053492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 356/728] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:21.057325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:21.179925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d56f66-36e9-4d19-ac52-df6d798cfaaf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:21.254985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:21.255945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:21.256237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:21.256525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:21.279903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:21.281961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:21.312629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:21.313698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:21.314663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:21.314663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:21.314663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:21.317485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437d98e-ca03-43b8-b868-d84051779567 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:21.318681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 373/729] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:21.325489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:21.407057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:21.407702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:21.407990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:21.408268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:21.408934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:21.409239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:21.410687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:21.434912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 357/730] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:21.450878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:21.455918 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:21.528701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:21.579716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b972825-6634-42c8-a8e7-d8774c38271b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:21.711143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:21.712102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:21.712593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:21.713077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:21.716560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:21.717178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:21.720675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:21.748791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b17056-f6f5-4fd3-b4a0-5984dd92f288 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:21.767320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 374/731] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:21.776710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:21.842310 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:21.844052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:21.844567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:21.845041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:21.863378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:21.866013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:21.915801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:21.916856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:21.917861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:21.918401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 358/732] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 468 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:21.947231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:21.998036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024837d2-3aaa-4d1f-8fc6-634f9347cb93 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:22.034459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:22.072411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a5c8c-5caf-4026-98b6-24d6df3a91c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:22.073598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a5c8c-5caf-4026-98b6-24d6df3a91c6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.074107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a5c8c-5caf-4026-98b6-24d6df3a91c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.074778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a5c8c-5caf-4026-98b6-24d6df3a91c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.091603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a5c8c-5caf-4026-98b6-24d6df3a91c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:22.095287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a5c8c-5caf-4026-98b6-24d6df3a91c6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:22.118995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2601022-bb5b-4f9e-b963-9071636b6e35 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-089a5c8c-5caf-4026-98b6-24d6df3a91c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:22.120528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:22.121571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:22.122540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:22.130099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:22.131219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 375/733] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:17:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:22.134229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc402e0-346b-427f-8492-1a39ff77b983 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.135045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc402e0-346b-427f-8492-1a39ff77b983 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.135810 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc402e0-346b-427f-8492-1a39ff77b983 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.138400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 376/734] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:22.139550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc402e0-346b-427f-8492-1a39ff77b983 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:22.168314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24567ac1-7c1e-47e4-9541-935203f9e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:22.168959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24567ac1-7c1e-47e4-9541-935203f9e91d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.169246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24567ac1-7c1e-47e4-9541-935203f9e91d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.169562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24567ac1-7c1e-47e4-9541-935203f9e91d None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.170318 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24567ac1-7c1e-47e4-9541-935203f9e91d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:22.170628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24567ac1-7c1e-47e4-9541-935203f9e91d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:22.172847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24567ac1-7c1e-47e4-9541-935203f9e91d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:22.206512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 359/735] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Oct 3 13:17:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:22.209529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906dd8b-6315-4539-a963-40ad7a1e581d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.210346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906dd8b-6315-4539-a963-40ad7a1e581d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.211148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906dd8b-6315-4539-a963-40ad7a1e581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.242605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6906dd8b-6315-4539-a963-40ad7a1e581d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:22.245063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6906dd8b-6315-4539-a963-40ad7a1e581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:22.280813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc402e0-346b-427f-8492-1a39ff77b983 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:22.354846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d82242-ee08-4a50-9029-1aa5338b4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:22.355904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d82242-ee08-4a50-9029-1aa5338b4ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.356403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d82242-ee08-4a50-9029-1aa5338b4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.356886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d82242-ee08-4a50-9029-1aa5338b4ea3 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.358356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d82242-ee08-4a50-9029-1aa5338b4ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:22.360057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d82242-ee08-4a50-9029-1aa5338b4ea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:22.362922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d82242-ee08-4a50-9029-1aa5338b4ea3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:22.379478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 360/736] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:22.379478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906dd8b-6315-4539-a963-40ad7a1e581d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:22.402832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 377/737] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Oct 3 13:17:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:22.405087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acac97-ebc2-4a36-830e-62e05dffa17f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.406191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acac97-ebc2-4a36-830e-62e05dffa17f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.406692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5acac97-ebc2-4a36-830e-62e05dffa17f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.414028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 378/738] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:22.415619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acac97-ebc2-4a36-830e-62e05dffa17f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:22.489274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:22.538558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:22.548828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906dd8b-6315-4539-a963-40ad7a1e581d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:22.594187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd11b2f-b4c3-4ee5-8ca3-84d54549bfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:22.595815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd11b2f-b4c3-4ee5-8ca3-84d54549bfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.596324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd11b2f-b4c3-4ee5-8ca3-84d54549bfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.596961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd11b2f-b4c3-4ee5-8ca3-84d54549bfd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.632201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5acac97-ebc2-4a36-830e-62e05dffa17f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:22.683205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1c2ed-0004-4f95-b622-3c75eb5bdb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:22.683824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1c2ed-0004-4f95-b622-3c75eb5bdb2d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.684105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1c2ed-0004-4f95-b622-3c75eb5bdb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.684377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1c2ed-0004-4f95-b622-3c75eb5bdb2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.711508 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd11b2f-b4c3-4ee5-8ca3-84d54549bfd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:22.714081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd11b2f-b4c3-4ee5-8ca3-84d54549bfd0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:22.792382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1c2ed-0004-4f95-b622-3c75eb5bdb2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:22.793813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1c2ed-0004-4f95-b622-3c75eb5bdb2d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:22.831856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:22.859147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:22.876648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 361/739] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:17:22.878731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366edd15-8aa1-480a-a757-263163e9675c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.879264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366edd15-8aa1-480a-a757-263163e9675c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.879767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366edd15-8aa1-480a-a757-263163e9675c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.902191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-366edd15-8aa1-480a-a757-263163e9675c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:22.903431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-366edd15-8aa1-480a-a757-263163e9675c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:22.916484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 379/740] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 03 13:17:22.917800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b543-0d00-4c6f-b130-895805250fdc None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:22.918112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b543-0d00-4c6f-b130-895805250fdc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:22.918362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b543-0d00-4c6f-b130-895805250fdc None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:22.919872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 380/741] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:22.925280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b543-0d00-4c6f-b130-895805250fdc None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:23.002155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 362/742] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:23.004038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366edd15-8aa1-480a-a757-263163e9675c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:23.060038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b543-0d00-4c6f-b130-895805250fdc None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:23.110387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8c28c-3cd1-4e7a-9a27-09f014d724ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:23.111449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8c28c-3cd1-4e7a-9a27-09f014d724ad None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:23.111972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8c28c-3cd1-4e7a-9a27-09f014d724ad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:23.112511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8c28c-3cd1-4e7a-9a27-09f014d724ad None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:23.113903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8c28c-3cd1-4e7a-9a27-09f014d724ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:23.117245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8c28c-3cd1-4e7a-9a27-09f014d724ad None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:23.126894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:23.149819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 381/743] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:17:22 2022] POST /identity/v3/services => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:23.156361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8c28c-3cd1-4e7a-9a27-09f014d724ad None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:23.182730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366edd15-8aa1-480a-a757-263163e9675c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:23.243700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:23.282010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7c2ac-859f-4738-9bcc-9d6c3030fa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:23.283302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7c2ac-859f-4738-9bcc-9d6c3030fa29 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:23.283721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7c2ac-859f-4738-9bcc-9d6c3030fa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:23.284223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7c2ac-859f-4738-9bcc-9d6c3030fa29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:23.308637 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7c2ac-859f-4738-9bcc-9d6c3030fa29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:23.313962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7c2ac-859f-4738-9bcc-9d6c3030fa29 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:23.316426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8c28c-3cd1-4e7a-9a27-09f014d724ad None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-7ef7c2ac-859f-4738-9bcc-9d6c3030fa29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:23.347412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:23.348481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:23.349383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:23.349383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:23.349383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:23.354085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 363/744] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:23.356633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6fc33-a616-4350-8f01-3d3eb4ab5b60 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:23.357205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6fc33-a616-4350-8f01-3d3eb4ab5b60 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:23.357897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6fc33-a616-4350-8f01-3d3eb4ab5b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:23.370950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:23.372088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:23.372523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:23.373009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:23.394990 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:23.397867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:23.400259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a5e6fc33-a616-4350-8f01-3d3eb4ab5b60 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:23.402736 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a5e6fc33-a616-4350-8f01-3d3eb4ab5b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:23.426728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:23.428804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:23.431185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:23.431185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:23.431185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:23.436154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 382/745] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:23.436154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:23.498545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:23.529509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 364/746] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 03 13:17:23.529509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6fc33-a616-4350-8f01-3d3eb4ab5b60 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:23.560502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb771a0-77fa-44b2-863c-b55816478908 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:23.610099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:23.610598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:23.610874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:23.611161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:23.611797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:23.612101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:23.613529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:23.633201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 383/747] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:23.633201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:23.739558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e705769-f3d2-4924-a897-c9c4558898f9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:23.745661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6fc33-a616-4350-8f01-3d3eb4ab5b60 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:23.749821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:23.752594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:23.782845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:23.783484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:23.783766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:23.784035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:23.784713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:23.785019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:23.787284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:23.800049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 384/748] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:23.800925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:23.826924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8aae9-80c7-486c-acb1-b6700f91814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:23.829124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8aae9-80c7-486c-acb1-b6700f91814c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:23.829667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8aae9-80c7-486c-acb1-b6700f91814c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:23.830154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8aae9-80c7-486c-acb1-b6700f91814c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:23.849296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8aae9-80c7-486c-acb1-b6700f91814c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-3ed8aae9-80c7-486c-acb1-b6700f91814c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:23.854643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:23.855761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:23.856606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:23.860222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 365/749] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:23.861323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8aae9-80c7-486c-acb1-b6700f91814c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:23.956450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c794a-2c46-4b90-baff-9b57c18889bd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:23.975000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8aae9-80c7-486c-acb1-b6700f91814c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:24.026346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61de68-8015-46bc-8612-f659b82ffa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:24.027387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61de68-8015-46bc-8612-f659b82ffa4a None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:24.027589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:24.027888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61de68-8015-46bc-8612-f659b82ffa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:24.028381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61de68-8015-46bc-8612-f659b82ffa4a None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:24.028480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:24.028889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:24.029298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:24.029598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61de68-8015-46bc-8612-f659b82ffa4a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:24.033685 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61de68-8015-46bc-8612-f659b82ffa4a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:24.042255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:24.045270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:24.065499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 366/750] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:17:23 2022] POST /identity/v3/services => generated 240 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:24.065499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61de68-8015-46bc-8612-f659b82ffa4a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:24.069170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:24.070517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:24.071418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:24.071418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:24.071418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:24.071418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:24.071418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:24.071418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:24.071418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:24.071418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:24.071860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 385/751] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:24.073305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:24.119903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:24.133139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:24.270845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61de68-8015-46bc-8612-f659b82ffa4a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:24.286797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fd8c4-57b5-4b89-8c95-9a6c53587a3b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:24.317492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:24.318446 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:24.318731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:24.319155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:24.342902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:24.346776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:24.366621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:24.367663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:24.368169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:24.368663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:24.370006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:24.370475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:24.373182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:24.377718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:24.379303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:24.381326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 367/752] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:24.383510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:24.393055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 386/753] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:24.394814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:24.553872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:24.598277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e5c92-908f-4c04-9e61-1af3d4787c5f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:24.612778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:24.633558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba276a45-a343-405b-b877-2e083afc5c4a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:24.664426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:24.665843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:24.666570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:24.667219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:24.668739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:24.669681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:24.674035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:24.699533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 368/754] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:24.699788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:24.700837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:24.701336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:24.701685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:24.703304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:24.703569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:24.704079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:24.707417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:24.738040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 387/755] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:24.739380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:24.898667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33fa2-a161-49cd-a45a-3032300d9077 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:24.937393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d044674-df8b-45ba-87ec-57d17b8c0e2a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:24.967817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:24.969739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:24.970603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:24.971380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:24.985951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:24.993590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:24.996897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:25.005883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:25.006847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:25.007294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:25.007842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:25.009126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=4ba4c42b47794c9ba5fc6ee557e2cfc3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:25.009679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:25.012298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:25.023532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 388/756] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:24 2022] GET /identity/v3/role_assignments?user.id=4ba4c42b47794c9ba5fc6ee557e2cfc3&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:25.025205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:25.036400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:25.038398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:25.040244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:25.040244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:25.040244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:25.043921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 369/757] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:25.053269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:25.058367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:25.161685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93e46-aa9b-42fb-9c04-cb5ef7e7a1ef None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:25.201379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:25.202111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:25.202398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:25.202820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:25.203573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c1ff08df377e4a83aae3afedd204e1ef) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:25.203881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:25.205409 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:25.213176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 389/758] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:25 2022] GET /identity/v3/role_assignments?user.id=c1ff08df377e4a83aae3afedd204e1ef&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:25.253995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71da21-bb14-4376-884f-261fdbc29674 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:25.311866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:25.312519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:25.312805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:25.313085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:25.313885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:25.314198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:25.315725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:25.335355 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 370/759] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:25.354647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:25.458542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:25.486817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b28e54-dedd-4796-881f-ed0b010c0b4d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:25.529576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:25.530354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:25.530646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:25.530923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:25.540049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:25.541668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:25.555671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:25.556729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:25.557742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 390/760] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:17:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:25.566714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:25.655293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:25.746367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c651-a061-41c1-be71-e819f99227d7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:25.824247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef0652-9a6e-4ef0-bd6e-24b214ccf008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:25.825292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef0652-9a6e-4ef0-bd6e-24b214ccf008 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:25.825790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef0652-9a6e-4ef0-bd6e-24b214ccf008 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:25.826275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef0652-9a6e-4ef0-bd6e-24b214ccf008 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:25.828843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef0652-9a6e-4ef0-bd6e-24b214ccf008 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:25.829390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef0652-9a6e-4ef0-bd6e-24b214ccf008 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:25.832141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef0652-9a6e-4ef0-bd6e-24b214ccf008 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:25.874863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 371/761] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:17:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:25.880444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:25.960272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774026-b07a-431f-89b7-b80d550e947e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:25.960781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774026-b07a-431f-89b7-b80d550e947e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:25.961285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df774026-b07a-431f-89b7-b80d550e947e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:25.968560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 372/762] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:17:25.977873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62ca9-afcf-4478-ad7a-5d5ac50abe6e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:25.978365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62ca9-afcf-4478-ad7a-5d5ac50abe6e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:25.978852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff62ca9-afcf-4478-ad7a-5d5ac50abe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:26.022572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4ff62ca9-afcf-4478-ad7a-5d5ac50abe6e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:26.025126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4ff62ca9-afcf-4478-ad7a-5d5ac50abe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:26.101519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ec920-467a-488d-9442-a4a4621306e7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:26.165332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:26.166360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:26.166535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:26.166881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:26.176357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:26.181683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:26.183752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:26.212182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:26.213506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:26.214880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:26.214880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:26.214880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:26.214880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 391/763] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:26.216593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 373/764] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:26.223230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:26.403036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10749703-ea54-416a-bb7e-d4cc3b1f95ed None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:26.459353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:26.459996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:26.460281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:26.460561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:26.461229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:26.461564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:26.462978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:26.488550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 392/765] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:26.494329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff62ca9-afcf-4478-ad7a-5d5ac50abe6e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:26.541159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:26.643786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff62ca9-afcf-4478-ad7a-5d5ac50abe6e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:26.693096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:26.694451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:26.694740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:26.695020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:26.718765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:26.725879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:26.743353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:26.744364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:26.745277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:26.745277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:26.745277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:26.745277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:26.745277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:26.745277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:26.745277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:26.745277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:26.748138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 374/766] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:26.759339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:26.796936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:26.946588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:27.084837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49f548-acb7-4ba2-a44b-8b7c4a9546de None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:27.095839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:27.103904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd0972-d619-492b-bafc-af41c8020059 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:27.180312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:27.184482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:27.185916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:27.186202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:27.188695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:27.189017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:27.190532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:27.205694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:27.207156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:27.208042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:27.208822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:27.210210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:27.213073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:27.230141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 393/767] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:27.231497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:27.234506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:27.236099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:27.237579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:27.241015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 375/768] 10.222.0.58 () {56 vars in 1129 bytes} [Mon Oct 3 13:17:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:27.250992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:27.403121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ad331-924c-4dd1-8dfd-6af59f9a6c05 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:27.454919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:27.455570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:27.455924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:27.456110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:27.456758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:27.457065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:27.458510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:27.475932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 376/769] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Oct 3 13:17:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:27.481223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:27.511463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6a3f5-7939-4a81-aed3-0671e9849a0f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:27.527940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:27.535463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:27.587435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a301409-a88f-461c-8cc1-2f448f5873fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:27.588772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a301409-a88f-461c-8cc1-2f448f5873fc None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:27.589340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a301409-a88f-461c-8cc1-2f448f5873fc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:27.589967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a301409-a88f-461c-8cc1-2f448f5873fc None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c3ece3405a2b43288136ddb79b84a106/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:27.595092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a301409-a88f-461c-8cc1-2f448f5873fc None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=c3ece3405a2b43288136ddb79b84a106, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:27.597999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a301409-a88f-461c-8cc1-2f448f5873fc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:27.625778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 394/770] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:27 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c3ece3405a2b43288136ddb79b84a106/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:17:27.627317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d414-a571-41a4-a78e-033681394984 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:27.627608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d414-a571-41a4-a78e-033681394984 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:27.627888 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d414-a571-41a4-a78e-033681394984 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:27.634717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 395/771] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:27.636084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651ea9-3594-494e-b3f8-63b3c2677579 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:27.636407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651ea9-3594-494e-b3f8-63b3c2677579 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:27.636636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2651ea9-3594-494e-b3f8-63b3c2677579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:27.659811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a2651ea9-3594-494e-b3f8-63b3c2677579 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:27.661410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a2651ea9-3594-494e-b3f8-63b3c2677579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:27.680062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c0c75-c5d9-4e1a-82c3-f860a3c242b4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:27.750216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 396/772] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:27.756737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:27.758330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:27.759178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:27.760004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:27.761616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:27.762453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:27.765934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:27.778930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 377/773] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Oct 3 13:17:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:27.785885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2651ea9-3594-494e-b3f8-63b3c2677579 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:27.931905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:27.936026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2651ea9-3594-494e-b3f8-63b3c2677579 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:27.979003 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:27.979628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:27.979905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:27.980190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:27.980892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] RBAC: Adding query filter params (interface=public, service_id=522aae6e67e54ace8e28a8a3d909f27b, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:27.981199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:27.982638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:27.992964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 397/774] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Oct 3 13:17:27 2022] GET /identity/v3/endpoints?service_id=522aae6e67e54ace8e28a8a3d909f27b&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:27.999994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:28.135365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:28.230255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:28.261402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc4ca5-0b7f-4672-ba21-ac528195b206 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:28.343117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd776176-3aae-4a7e-a51c-54e0bf4bd334 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:28.352679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a19a1-6ffb-4bff-9a3a-47d5297f7ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:28.353381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a19a1-6ffb-4bff-9a3a-47d5297f7ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:28.353694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a19a1-6ffb-4bff-9a3a-47d5297f7ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:28.353968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a19a1-6ffb-4bff-9a3a-47d5297f7ba2 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:28.354599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a19a1-6ffb-4bff-9a3a-47d5297f7ba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:28.356140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a19a1-6ffb-4bff-9a3a-47d5297f7ba2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:28.372364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 378/775] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:17:27 2022] GET /identity/v3/services => generated 772 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:28.395657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c3be6-d97f-41bd-b48d-d56fb791127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:28.396628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3be6-d97f-41bd-b48d-d56fb791127e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:28.397094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3be6-d97f-41bd-b48d-d56fb791127e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:28.397591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3be6-d97f-41bd-b48d-d56fb791127e None admin] PATH_INFO: `/v3/services/network` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:28.398398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c3be6-d97f-41bd-b48d-d56fb791127e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:28.399983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91c3be6-d97f-41bd-b48d-d56fb791127e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b91c3be6-d97f-41bd-b48d-d56fb791127e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:28.425433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:28.426705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:28.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:28.430884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 398/776] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:28 2022] GET /identity/v3/services/network => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:28.444096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a19a1-6ffb-4bff-9a3a-47d5297f7ba2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:28.551158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e30e9-d87f-409f-be73-17dc1d04afd5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:28.551836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e30e9-d87f-409f-be73-17dc1d04afd5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:28.552382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e30e9-d87f-409f-be73-17dc1d04afd5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:28.554287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 399/777] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:28.562666 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c6b52-5344-4700-b42c-57070c614b6b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:28.563237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c6b52-5344-4700-b42c-57070c614b6b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:28.563740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c6b52-5344-4700-b42c-57070c614b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:28.625382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a72c6b52-5344-4700-b42c-57070c614b6b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:28.628911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a72c6b52-5344-4700-b42c-57070c614b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:28.659349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:28.679463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a19a1-6ffb-4bff-9a3a-47d5297f7ba2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:28.737095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a87f2-05ac-46e8-8e21-d6931f18354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:28.738157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a87f2-05ac-46e8-8e21-d6931f18354a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:28.738157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a87f2-05ac-46e8-8e21-d6931f18354a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:28.738412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a87f2-05ac-46e8-8e21-d6931f18354a None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:28.738954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a87f2-05ac-46e8-8e21-d6931f18354a None admin] RBAC: Adding query filter params (name=network) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:28.739255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a87f2-05ac-46e8-8e21-d6931f18354a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:28.740644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a87f2-05ac-46e8-8e21-d6931f18354a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:28.746344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 379/778] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:28.749813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7bcd-7e15-4120-ba9e-ac045e06ac87 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:28.749813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7bcd-7e15-4120-ba9e-ac045e06ac87 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:28.749813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f7bcd-7e15-4120-ba9e-ac045e06ac87 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:28.751404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 380/779] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:28.754567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f7bcd-7e15-4120-ba9e-ac045e06ac87 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:28.776497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:28.777963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 400/780] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:28.781022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41c19-6915-467f-a041-5d74d2aaa06c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:28.781934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41c19-6915-467f-a041-5d74d2aaa06c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:28.783376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41c19-6915-467f-a041-5d74d2aaa06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:28.804030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-35f41c19-6915-467f-a041-5d74d2aaa06c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:28.805720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-35f41c19-6915-467f-a041-5d74d2aaa06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:28.887411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f7bcd-7e15-4120-ba9e-ac045e06ac87 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:28.921679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 401/781] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:28.956998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:28.957654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:28.957936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:28.958212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:28.958921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] RBAC: Adding query filter params (type=network) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:28.959227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:28.960678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:28.966746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 381/782] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:28.976543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f41c19-6915-467f-a041-5d74d2aaa06c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:29.100389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:29.117735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f41c19-6915-467f-a041-5d74d2aaa06c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:29.159710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:29.160581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:29.161017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:29.161443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:29.162348 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] RBAC: Adding query filter params (interface=public, service_id=136b3feab0b848b3b3f8322de9efd633, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:29.162813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:29.164344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:29.170511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 402/783] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Oct 3 13:17:28 2022] GET /identity/v3/endpoints?service_id=136b3feab0b848b3b3f8322de9efd633&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:29.179326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:29.264045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:29.385843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:29.410706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16c4e2-0b25-4fec-9c3d-587227876255 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:29.411269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311e1a1-ca91-4125-bf71-ed0c42c1ad3e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:29.449211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a246498-69cb-4dcc-9f8c-9dac96d5445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:29.449492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ac14d-7dd0-414c-8bff-73c90bcf9e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:29.449899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ac14d-7dd0-414c-8bff-73c90bcf9e9a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:29.450187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ac14d-7dd0-414c-8bff-73c90bcf9e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:29.450307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a246498-69cb-4dcc-9f8c-9dac96d5445e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:29.450465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ac14d-7dd0-414c-8bff-73c90bcf9e9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:29.450731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a246498-69cb-4dcc-9f8c-9dac96d5445e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:29.451097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a246498-69cb-4dcc-9f8c-9dac96d5445e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:29.451809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a246498-69cb-4dcc-9f8c-9dac96d5445e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:29.454084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a246498-69cb-4dcc-9f8c-9dac96d5445e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:29.460407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 382/784] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:17:29 2022] GET /identity/v3/services => generated 772 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:29.461374 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a246498-69cb-4dcc-9f8c-9dac96d5445e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:29.463730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ac14d-7dd0-414c-8bff-73c90bcf9e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-344ac14d-7dd0-414c-8bff-73c90bcf9e9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:29.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:29.472147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:29.473238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 403/785] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:29.479836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ac14d-7dd0-414c-8bff-73c90bcf9e9a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:29.514112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:29.650012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ac14d-7dd0-414c-8bff-73c90bcf9e9a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:29.701940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:29.703477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:29.703959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:29.704424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:29.705285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:29.705792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:29.705994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a246498-69cb-4dcc-9f8c-9dac96d5445e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:29.707713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:29.719117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 404/786] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:29.724808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:29.761631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f279f1-5717-4330-a705-5c23cea120a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:29.763061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f279f1-5717-4330-a705-5c23cea120a1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:29.763571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f279f1-5717-4330-a705-5c23cea120a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:29.764061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f279f1-5717-4330-a705-5c23cea120a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:29.776745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f279f1-5717-4330-a705-5c23cea120a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-62f279f1-5717-4330-a705-5c23cea120a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:29.783312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:29.784302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:29.785136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:29.788148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 383/787] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:29.794748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f279f1-5717-4330-a705-5c23cea120a1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:29.805677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:29.825314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:29.856966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f29eb-f6a0-4429-a510-c2a7fba70e1a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:29.916267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f279f1-5717-4330-a705-5c23cea120a1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:29.919423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:29.920269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:29.921299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:29.925520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:29.951676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:29.954355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:29.956326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:29.961341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:29.961341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:29.961341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:29.961341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:29.961341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:29.961341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:29.970385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 384/788] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:29.976174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:29.989564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:29.991685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:29.993584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:29.993584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:29.993584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:29.997506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 405/789] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:30.006805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:30.055100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:30.096168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc193d-d1fe-440f-8730-81d53b0d08bf None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:30.163065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:30.163745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:30.164030 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:30.164329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:30.191182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:30.192679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:30.226214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:30.228129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:30.230593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:30.230593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:30.230593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:30.230593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 385/790] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:30.241404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:30.291800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee368bc-36b6-41ee-96b7-df0f4f9f8c10 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:30.361502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:30.373494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:30.375784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:30.376072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:30.376339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:30.377013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:30.377313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:30.378778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:30.413169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 406/791] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:30.423923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:30.447249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1d319-e729-4440-8fa8-95ce6dc072fd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:30.470454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:30.516761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:30.517704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:30.518192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:30.518673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:30.519846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:30.520392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:30.522925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:30.576832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 386/792] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:30.580922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:30.774181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a369c04-147e-461e-922a-ce3455ff9b4d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:30.799638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fb428-6b44-48ed-85fd-abc6c09203f4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:30.865974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:30.867084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:30.867586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:30.868133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:30.883861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:30.886836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:30.912386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:30.914090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:30.915762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 387/793] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:17:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:30.919810 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:30.921404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:30.921945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:30.922447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:30.926572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:30.940540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:30.947303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:30.970911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:30.973141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:30.975080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:30.981252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 407/794] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 555 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:30.985773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:31.026058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:31.069862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:31.147846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557082a-6585-4ba9-ba32-f45125d4fee1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:31.184159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bf110-f409-41e6-a0b0-9eb643aba81f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:31.222000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:31.222642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:31.222929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:31.223257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:31.224017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:31.224334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:31.226085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:31.251961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:31.252595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:31.252875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:31.253156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:31.253961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:31.254278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:31.256019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:31.264002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 388/795] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Oct 3 13:17:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:31.270621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:31.284228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 408/796] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Oct 3 13:17:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:31.290202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:31.399938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:31.404656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2db8b5-70d8-4a85-93af-8c3e9322fad3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:31.455968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9229ffd-9e28-4da0-97bb-13aec40cc00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:31.456832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9229ffd-9e28-4da0-97bb-13aec40cc00a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:31.457208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9229ffd-9e28-4da0-97bb-13aec40cc00a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:31.457616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9229ffd-9e28-4da0-97bb-13aec40cc00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:31.474369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9229ffd-9e28-4da0-97bb-13aec40cc00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:31.476128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9229ffd-9e28-4da0-97bb-13aec40cc00a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b9229ffd-9e28-4da0-97bb-13aec40cc00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:31.494727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:31.496928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:31.499525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:31.499525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:31.499525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:31.499525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 389/797] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:31.505869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5bc1a-c077-4175-8db2-c8eeb9f37517 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:31.506303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5bc1a-c077-4175-8db2-c8eeb9f37517 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:31.506696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb5bc1a-c077-4175-8db2-c8eeb9f37517 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:31.513568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 390/798] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:31.513568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5bc1a-c077-4175-8db2-c8eeb9f37517 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:31.571133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d0d18-91c1-4fb0-88ce-73e5e82323ce None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:31.577817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:31.634866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978f626-6629-4be2-aa23-8805568d7a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:31.635783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978f626-6629-4be2-aa23-8805568d7a65 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:31.636165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978f626-6629-4be2-aa23-8805568d7a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:31.636546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978f626-6629-4be2-aa23-8805568d7a65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:31.656051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978f626-6629-4be2-aa23-8805568d7a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:31.661847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978f626-6629-4be2-aa23-8805568d7a65 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:31.662712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb5bc1a-c077-4175-8db2-c8eeb9f37517 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f978f626-6629-4be2-aa23-8805568d7a65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:31.699923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:31.701005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:31.702371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:31.702371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:31.702371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:31.702928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 409/799] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:31.706175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102ba57-a598-4b52-bb52-e643647b8d5f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:31.706628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102ba57-a598-4b52-bb52-e643647b8d5f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:31.707102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102ba57-a598-4b52-bb52-e643647b8d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:31.710789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:31.711742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:31.712333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:31.712811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:31.713781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:31.714356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:31.716018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:31.734482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 391/800] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:31.735694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:31.743599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6102ba57-a598-4b52-bb52-e643647b8d5f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:31.745234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6102ba57-a598-4b52-bb52-e643647b8d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:31.759201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:31.866766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 410/801] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:31.875264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102ba57-a598-4b52-bb52-e643647b8d5f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:31.901924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b397394-abd7-45cf-b22b-2030507f3ae9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:31.957623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:31.958683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:31.961899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:31.964096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:31.965203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:31.965800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:31.969098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:32.019957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 392/802] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:32.026085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:32.110824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:32.132357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102ba57-a598-4b52-bb52-e643647b8d5f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:32.182650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6dc564-6fb3-407f-a56d-950f992cdbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:32.184060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dc564-6fb3-407f-a56d-950f992cdbbe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.184674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dc564-6fb3-407f-a56d-950f992cdbbe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.185433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6dc564-6fb3-407f-a56d-950f992cdbbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.195329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6dc564-6fb3-407f-a56d-950f992cdbbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:32.197635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6dc564-6fb3-407f-a56d-950f992cdbbe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:32.199006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5f6dc564-6fb3-407f-a56d-950f992cdbbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:32.223075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:32.224097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:32.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:32.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:32.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:32.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:32.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:32.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:32.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:32.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:32.230140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 411/803] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:32.232479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef97593-9111-472e-bf40-7a1fae730098 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.233980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef97593-9111-472e-bf40-7a1fae730098 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.234733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef97593-9111-472e-bf40-7a1fae730098 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.240274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 412/804] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:32.241873 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef97593-9111-472e-bf40-7a1fae730098 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:32.255344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd149e-91b2-4f54-89e8-66b0a1c53bd3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:32.310301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0547d1d-dbb0-4e36-a15c-c99e6629e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:32.311844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0547d1d-dbb0-4e36-a15c-c99e6629e443 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.312636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0547d1d-dbb0-4e36-a15c-c99e6629e443 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.313435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0547d1d-dbb0-4e36-a15c-c99e6629e443 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.327457 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0547d1d-dbb0-4e36-a15c-c99e6629e443 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:32.330760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0547d1d-dbb0-4e36-a15c-c99e6629e443 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c0547d1d-dbb0-4e36-a15c-c99e6629e443 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:32.358396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:32.359757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 393/805] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50053-fb08-44a3-8fb0-b72c4ed369d3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.365747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50053-fb08-44a3-8fb0-b72c4ed369d3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.366372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50053-fb08-44a3-8fb0-b72c4ed369d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.394069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f0f50053-fb08-44a3-8fb0-b72c4ed369d3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:32.394383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef97593-9111-472e-bf40-7a1fae730098 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:32.396012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f0f50053-fb08-44a3-8fb0-b72c4ed369d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:32.436879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015845cd-184a-4168-929a-433e782ea1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:32.437533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015845cd-184a-4168-929a-433e782ea1da None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.437823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015845cd-184a-4168-929a-433e782ea1da None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.438104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015845cd-184a-4168-929a-433e782ea1da None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.438841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015845cd-184a-4168-929a-433e782ea1da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:32.439142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015845cd-184a-4168-929a-433e782ea1da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:32.440632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015845cd-184a-4168-929a-433e782ea1da None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:32.455039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 413/806] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:32.456387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066176d3-99bf-4f22-ba65-b702941ec838 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.456676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066176d3-99bf-4f22-ba65-b702941ec838 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.456957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066176d3-99bf-4f22-ba65-b702941ec838 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.458345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 414/807] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:32.459115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066176d3-99bf-4f22-ba65-b702941ec838 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:32.470112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:32.511462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 394/808] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:32.512793 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f50053-fb08-44a3-8fb0-b72c4ed369d3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:32.594894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066176d3-99bf-4f22-ba65-b702941ec838 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:32.613008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:32.651366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:32.652448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.652877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.653383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.654653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:32.655651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:32.658870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:32.669035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f50053-fb08-44a3-8fb0-b72c4ed369d3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:32.687092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 415/809] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:32.689777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:32.710246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38230212-634e-4cfd-b63c-54551174f99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:32.710901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38230212-634e-4cfd-b63c-54551174f99f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.711187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38230212-634e-4cfd-b63c-54551174f99f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.711465 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38230212-634e-4cfd-b63c-54551174f99f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.725085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38230212-634e-4cfd-b63c-54551174f99f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-38230212-634e-4cfd-b63c-54551174f99f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:32.729999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:32.731830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 395/810] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:32.733575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5c53-656d-42bd-8abd-3ccaef6f6601 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.735101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5c53-656d-42bd-8abd-3ccaef6f6601 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.735101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5c53-656d-42bd-8abd-3ccaef6f6601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.760924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cf8b5c53-656d-42bd-8abd-3ccaef6f6601 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:32.763036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cf8b5c53-656d-42bd-8abd-3ccaef6f6601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:32.799708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:32.840768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081c31a-fe71-47a2-a311-223c4fb60806 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:32.872968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 396/811] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:32.873890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b5c53-656d-42bd-8abd-3ccaef6f6601 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:32.895749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25a06f-4b0c-44dc-b14d-1d6235db52bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:32.897001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25a06f-4b0c-44dc-b14d-1d6235db52bc None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:32.897331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25a06f-4b0c-44dc-b14d-1d6235db52bc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:32.897676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25a06f-4b0c-44dc-b14d-1d6235db52bc None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c1ff08df377e4a83aae3afedd204e1ef/roles/c49c853797c4458d84ca51c515a08755` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:32.926679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25a06f-4b0c-44dc-b14d-1d6235db52bc None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=c1ff08df377e4a83aae3afedd204e1ef, role_id=c49c853797c4458d84ca51c515a08755)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:32.929336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25a06f-4b0c-44dc-b14d-1d6235db52bc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:32.941986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:32.965483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 416/812] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:32 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c1ff08df377e4a83aae3afedd204e1ef/roles/c49c853797c4458d84ca51c515a08755 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:17:32.965483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25a06f-4b0c-44dc-b14d-1d6235db52bc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:33.039252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b5c53-656d-42bd-8abd-3ccaef6f6601 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:33.085943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c00029-ed0d-4834-a708-375ac05f9494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:33.086734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c00029-ed0d-4834-a708-375ac05f9494 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:33.087328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c00029-ed0d-4834-a708-375ac05f9494 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:33.087686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c00029-ed0d-4834-a708-375ac05f9494 None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/4ba4c42b47794c9ba5fc6ee557e2cfc3/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:33.090635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c00029-ed0d-4834-a708-375ac05f9494 None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=4ba4c42b47794c9ba5fc6ee557e2cfc3, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:33.092270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c00029-ed0d-4834-a708-375ac05f9494 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:33.112716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 397/813] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:32 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/4ba4c42b47794c9ba5fc6ee557e2cfc3/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:17:33.117430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c00029-ed0d-4834-a708-375ac05f9494 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:33.163813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25a06f-4b0c-44dc-b14d-1d6235db52bc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:33.199033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:33.236927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:33.238403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:33.239114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:33.239895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:33.241372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:33.242216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:33.245292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:33.253502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c00029-ed0d-4834-a708-375ac05f9494 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:33.266567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 417/814] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:33.268615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:33.268889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:33.305157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:33.305616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:33.305908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:33.306189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:33.306816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:33.308267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:33.320091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:33.322148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:33.323888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 398/815] 10.222.0.58 () {56 vars in 1129 bytes} [Mon Oct 3 13:17:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:33.326576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:33.411766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1e3d6-009c-4188-b85d-54a29780c80b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:33.452788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:33.453555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:33.453847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:33.454160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] PATH_INFO: `/v3/services/network` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:33.454926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:33.456358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:33.466534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:33.469076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:33.470849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 418/816] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:33 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:33.474158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:33.485055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f6030-1f7e-491b-a9ee-10a4a09e70d0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:33.538271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:33.566178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:33.567624 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:33.568336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:33.569314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:33.598719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:33.600593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:33.621753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:33.625239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:33.628349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:33.628349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:33.628349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:33.637493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 399/817] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:33.637493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:33.638143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d44df-866a-4421-9793-8c1f4e34ca7c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:33.676189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:33.707957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:33.708551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:33.708839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:33.709474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:33.710156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:33.710468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:33.712594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:33.719744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 419/818] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Oct 3 13:17:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:33.720769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:33.827256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a69de-aaae-4ad3-9098-406102f82fac None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:33.877822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64225add-5675-4246-8e3d-939ec9785005 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:33.896773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:33.899728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:33.900905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:33.901298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:33.901931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:33.903158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:33.903645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:33.907284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:33.917111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 400/819] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:33.923172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:33.965375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:33.966637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:33.967071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:33.967754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:33.969025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:33.969749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:33.971380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:33.989801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 420/820] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:33.990945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:34.117055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:34.121008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaeb0eb-cd58-4877-ab57-154098dcece4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:34.137773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89aff6-a346-40b1-a37d-8315d57b60b7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:34.165372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:34.166362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:34.166852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:34.167405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:34.168282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] RBAC: Adding query filter params (type=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:34.168813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:34.170597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:34.178028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 421/821] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:34.179298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:34.216368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:34.217368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:34.218023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:34.218498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:34.218635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:34.219727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:34.220342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:34.223293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:34.233633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 401/822] 10.222.0.58 () {56 vars in 1124 bytes} [Mon Oct 3 13:17:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:34.235687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:34.310088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd59f6e-37a0-4e79-9070-ade1bfa62164 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:34.352681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:34.353762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:34.354355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:34.354836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:34.366044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:34.367894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:34.382928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:34.384051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:17:34.385023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:34.387743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 422/823] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:17:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:34.388828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:34.449220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b1112-0af9-4533-82f4-298aa2d3aba9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:34.503985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:34.516796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418908b-d155-4f85-b638-42749647b6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:34.517901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418908b-d155-4f85-b638-42749647b6b3 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:34.518401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418908b-d155-4f85-b638-42749647b6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:34.518992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ded4e0-ed8a-497e-8520-5b4f398ca345 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:34.519105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418908b-d155-4f85-b638-42749647b6b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:34.520223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418908b-d155-4f85-b638-42749647b6b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:34.522913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418908b-d155-4f85-b638-42749647b6b3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:34.561772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b7911-08aa-4560-a118-89d91ff85f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:34.563157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b7911-08aa-4560-a118-89d91ff85f56 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:34.563662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b7911-08aa-4560-a118-89d91ff85f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:34.564142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b7911-08aa-4560-a118-89d91ff85f56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:34.565181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b7911-08aa-4560-a118-89d91ff85f56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:34.567005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b7911-08aa-4560-a118-89d91ff85f56 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:34.570683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 402/824] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:17:34 2022] POST /identity/v3/endpoints => generated 337 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:34.574198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418908b-d155-4f85-b638-42749647b6b3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:34.606128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 423/825] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:17:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:34.625383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:34.831109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418908b-d155-4f85-b638-42749647b6b3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:34.922985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:34.922985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:34.923312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:34.923588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:34.925901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:34.925901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:34.928316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:34.969622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 403/826] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:17:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:34.976272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b7911-08aa-4560-a118-89d91ff85f56 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:35.160188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b7911-08aa-4560-a118-89d91ff85f56 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:35.211584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549113e3-db30-4286-b06d-751f18186c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:35.212608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549113e3-db30-4286-b06d-751f18186c54 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:35.213103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549113e3-db30-4286-b06d-751f18186c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:35.213605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549113e3-db30-4286-b06d-751f18186c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:35.231433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:35.302165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549113e3-db30-4286-b06d-751f18186c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:35.304045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549113e3-db30-4286-b06d-751f18186c54 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:35.424836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 424/827] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:34 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:17:35.431038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:35.501885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:35.552446 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a951247-5f0c-4793-a468-b7dff0a92c45 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:35.596758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95774e8-396d-4186-88e8-83d3c2ad2941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:35.597443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95774e8-396d-4186-88e8-83d3c2ad2941 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:35.597756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95774e8-396d-4186-88e8-83d3c2ad2941 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:35.598031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95774e8-396d-4186-88e8-83d3c2ad2941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:35.615753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95774e8-396d-4186-88e8-83d3c2ad2941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:35.617686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95774e8-396d-4186-88e8-83d3c2ad2941 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b95774e8-396d-4186-88e8-83d3c2ad2941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:35.637596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:35.638716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:35.640676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:35.640676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:35.640676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:35.643306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 404/828] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:35.653074 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549113e3-db30-4286-b06d-751f18186c54 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:35.705565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ce7d3-e428-4742-a14c-193560ddfd47 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:35.706239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ce7d3-e428-4742-a14c-193560ddfd47 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:35.706730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ce7d3-e428-4742-a14c-193560ddfd47 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:35.709351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 405/829] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:17:35.716234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122cd5-6ce0-483e-86a3-768170a8dffd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:35.716881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122cd5-6ce0-483e-86a3-768170a8dffd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:35.717477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9122cd5-6ce0-483e-86a3-768170a8dffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:35.719685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:35.739583 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b9122cd5-6ce0-483e-86a3-768170a8dffd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:35.741295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b9122cd5-6ce0-483e-86a3-768170a8dffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:35.779756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549113e3-db30-4286-b06d-751f18186c54 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:35.838035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 406/830] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:35.846968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450c1ef-f5ea-4798-a4ac-1fa26d846d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:35.848026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450c1ef-f5ea-4798-a4ac-1fa26d846d22 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:35.848542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450c1ef-f5ea-4798-a4ac-1fa26d846d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:35.849029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450c1ef-f5ea-4798-a4ac-1fa26d846d22 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:35.850310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450c1ef-f5ea-4798-a4ac-1fa26d846d22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:35.850878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450c1ef-f5ea-4798-a4ac-1fa26d846d22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:35.853592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450c1ef-f5ea-4798-a4ac-1fa26d846d22 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:35.879154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 425/831] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:35.888552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122cd5-6ce0-483e-86a3-768170a8dffd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:35.977044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:36.070625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9122cd5-6ce0-483e-86a3-768170a8dffd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:36.104137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a800fa-cde3-45fe-85db-e1a82bd7b8a5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:36.104694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a800fa-cde3-45fe-85db-e1a82bd7b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:36.105261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a800fa-cde3-45fe-85db-e1a82bd7b8a5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:36.113752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 426/832] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:36.117642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:36.117930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a40a-5751-47da-bed6-65cdbc078106 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:36.118429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a40a-5751-47da-bed6-65cdbc078106 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:36.118931 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8a40a-5751-47da-bed6-65cdbc078106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:36.119209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:36.119689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:36.120159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:36.128502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:36.130755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:36.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:36.145621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:36.146702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:36.146702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:36.146702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:36.146702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:36.146702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:36.146702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:36.146702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:36.146702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:36.148907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 407/833] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:36.153505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-28d8a40a-5751-47da-bed6-65cdbc078106 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:36.157487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-28d8a40a-5751-47da-bed6-65cdbc078106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:36.159335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:36.271557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 427/834] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:36.279219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8a40a-5751-47da-bed6-65cdbc078106 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:36.371766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:36.394845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01733b0-da5b-441b-9221-464664661d0c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:36.398347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:36.478793 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:36.479917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:36.480369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:36.480943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:36.482307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:36.483839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:36.487562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:36.510427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8a40a-5751-47da-bed6-65cdbc078106 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:36.521440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 408/835] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:36.524413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:36.575720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209cc98-0b12-4ac9-8b2b-8777b91e175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:36.577509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209cc98-0b12-4ac9-8b2b-8777b91e175d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:36.577509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209cc98-0b12-4ac9-8b2b-8777b91e175d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:36.577862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209cc98-0b12-4ac9-8b2b-8777b91e175d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:36.598368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209cc98-0b12-4ac9-8b2b-8777b91e175d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2209cc98-0b12-4ac9-8b2b-8777b91e175d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:36.607176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:36.610237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:17:36.613143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:36.618660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 428/836] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:17:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:36.620108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209cc98-0b12-4ac9-8b2b-8777b91e175d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:36.725123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa03bc-40f0-4998-91c4-b8c2bef8c3a5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:36.798147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:36.799657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:36.799876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:36.800408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:36.801025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:36.803239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c3ece3405a2b43288136ddb79b84a106) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:36.803975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:36.806812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:36.809161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209cc98-0b12-4ac9-8b2b-8777b91e175d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:36.819623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 409/837] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:36 2022] GET /identity/v3/role_assignments?user.id=c3ece3405a2b43288136ddb79b84a106&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:36.825711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:36.881330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefe37f-be8e-4198-9afa-b31e4d41c5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:36.882887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefe37f-be8e-4198-9afa-b31e4d41c5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:36.882887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefe37f-be8e-4198-9afa-b31e4d41c5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:36.883295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefe37f-be8e-4198-9afa-b31e4d41c5f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:36.896865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:36.905027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefe37f-be8e-4198-9afa-b31e4d41c5f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-afefe37f-be8e-4198-9afa-b31e4d41c5f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:36.914183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:36.916032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:17:36.917646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:36.923058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 429/838] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:36.926263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefe37f-be8e-4198-9afa-b31e4d41c5f8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:37.064317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bee121-fb8e-4e05-b0ea-d7e65c075b14 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:37.135768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:37.136537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.136823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.137098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.137822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:37.138126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:37.139625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:37.149948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 410/839] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:17:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:37.154065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefe37f-be8e-4198-9afa-b31e4d41c5f8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:37.159040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:37.223240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:37.224461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.224995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.225663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.226743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:37.227328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:37.229732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:37.243286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 430/840] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:37.252753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:37.275834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:37.392212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c3182-dbc8-4274-a90b-6ea1f3d9118b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:37.406268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:37.423566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91451ecb-12e8-487b-88f8-3f00e66716de None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:37.443585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:37.444237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.444518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.444800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.460135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:37.462329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:37.471990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:37.473715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.474584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.475395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:37.480655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:37.483766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:37.486262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:37.486262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:37.486262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:37.487690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 431/841] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:37.489440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:37.506966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:37.509392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:37.531745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:37.532787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:37.533714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:37.533714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:37.533714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:37.535901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 411/842] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:37.545872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:37.596598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:37.641254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535c563-5a7d-46ce-b453-075cae73b02e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:37.682079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c11ba3-6891-4f88-98b6-9eda780a9b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:37.682692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c11ba3-6891-4f88-98b6-9eda780a9b8a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.682988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c11ba3-6891-4f88-98b6-9eda780a9b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.683238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c11ba3-6891-4f88-98b6-9eda780a9b8a None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.683886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c11ba3-6891-4f88-98b6-9eda780a9b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:37.684191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c11ba3-6891-4f88-98b6-9eda780a9b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:37.685624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c11ba3-6891-4f88-98b6-9eda780a9b8a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:37.698896 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 432/843] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:37.700183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e943ac-0b7f-4a56-a8df-eeeb86e9d8f9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.700468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e943ac-0b7f-4a56-a8df-eeeb86e9d8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.700733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e943ac-0b7f-4a56-a8df-eeeb86e9d8f9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.702993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 433/844] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:37.708863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e943ac-0b7f-4a56-a8df-eeeb86e9d8f9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:37.727108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d77fcd-6434-4c8a-9445-b6d7a04f51d9 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:37.786023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c03f82-0917-4dda-837b-3eae21b19a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:37.786992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c03f82-0917-4dda-837b-3eae21b19a47 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.787453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c03f82-0917-4dda-837b-3eae21b19a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.788003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c03f82-0917-4dda-837b-3eae21b19a47 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.788883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c03f82-0917-4dda-837b-3eae21b19a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:37.789765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c03f82-0917-4dda-837b-3eae21b19a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:37.791495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c03f82-0917-4dda-837b-3eae21b19a47 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:37.805836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:37.811551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 412/845] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:37.813783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58ab38-d794-4e99-ae11-1aedb40a0990 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.814606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58ab38-d794-4e99-ae11-1aedb40a0990 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.815375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58ab38-d794-4e99-ae11-1aedb40a0990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.846147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5c58ab38-d794-4e99-ae11-1aedb40a0990 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:37.848049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5c58ab38-d794-4e99-ae11-1aedb40a0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:37.860840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e943ac-0b7f-4a56-a8df-eeeb86e9d8f9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:37.913864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:37.930294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:37.931257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:37.931713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:37.932176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:37.946229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:37.948772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:37.959619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 413/846] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:37.960709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58ab38-d794-4e99-ae11-1aedb40a0990 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:37.980880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:37.982386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 03 13:17:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:37.991031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 434/847] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:17:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:37.992494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:38.136566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58ab38-d794-4e99-ae11-1aedb40a0990 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:38.155300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758b6f7-ceaa-452d-8f3b-9d04c86ec46d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:38.176258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:38.185348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:38.186831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:38.187114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:38.187388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:38.199946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:38.203397 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:38.221486 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:38.222392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:38.222842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:38.223317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:38.224273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:38.224733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:38.225943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:38.227067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:38.228193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:38.228756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 414/848] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:17:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:38.229681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:38.240018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:38.269805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 435/849] 10.222.0.58 () {56 vars in 1135 bytes} [Mon Oct 3 13:17:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:38.281537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:38.377606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:38.402796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5f004-345c-4e8b-87de-1425433ef44c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:38.446475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:38.447399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:38.447964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:38.448381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:38.449419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:38.449761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:38.451248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:38.469886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc15be67-0011-428d-a195-abed0fe37737 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:38.473554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 415/850] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Oct 3 13:17:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:38.474805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:38.522233 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560e4ff-56ec-493c-bd8e-e3d191e52135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:38.522870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560e4ff-56ec-493c-bd8e-e3d191e52135 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:38.523199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560e4ff-56ec-493c-bd8e-e3d191e52135 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:38.523487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560e4ff-56ec-493c-bd8e-e3d191e52135 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:38.562408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:38.611869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560e4ff-56ec-493c-bd8e-e3d191e52135 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:38.613778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560e4ff-56ec-493c-bd8e-e3d191e52135 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:38.640261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca6d0-d3be-4b56-b7df-4ff96ed79c96 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:38.692463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:38.704342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:38.705436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:38.705855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:38.706179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:38.707167 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:38.709561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:38.722585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:38.723953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:38.725027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 416/851] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:38.726132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:38.784796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 436/852] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:17:38.786319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560e4ff-56ec-493c-bd8e-e3d191e52135 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:38.875649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd79403-4c90-43d3-bc21-720b7ddce61e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:38.892105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:38.925696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93fd85-4535-4ae9-8a7a-c9ad79ed8d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:38.926787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93fd85-4535-4ae9-8a7a-c9ad79ed8d88 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:38.927287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93fd85-4535-4ae9-8a7a-c9ad79ed8d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:38.927778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93fd85-4535-4ae9-8a7a-c9ad79ed8d88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:38.934048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560e4ff-56ec-493c-bd8e-e3d191e52135 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:38.973669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:38.974201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:38.974485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:38.974922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:38.975387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:38.975689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:38.977068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:38.981935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 437/853] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:38.983353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:39.001545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:39.057650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93fd85-4535-4ae9-8a7a-c9ad79ed8d88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:39.060288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93fd85-4535-4ae9-8a7a-c9ad79ed8d88 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:39.103326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b93909-66d1-4f9a-9beb-e68057b24ed6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:39.160976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:39.162211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:39.162661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:39.163122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:39.177047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:39.178688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:39.195856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:39.197332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:39.198633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:39.198633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:39.198633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:39.198633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 438/854] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:39.198897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:39.228068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 417/855] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:17:39.229323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93fd85-4535-4ae9-8a7a-c9ad79ed8d88 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:39.247217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:39.298303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:39.382442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462c241-9516-402b-9cce-446e08acdd5f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:39.392413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93fd85-4535-4ae9-8a7a-c9ad79ed8d88 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:39.447248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:39.447902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:39.448222 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:39.448470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:39.449141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:39.449480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:39.451095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:39.469494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 439/856] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:39.469494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:39.473755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370dc50-bd18-481c-bc67-00c64142843d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:39.473755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370dc50-bd18-481c-bc67-00c64142843d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:39.473755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370dc50-bd18-481c-bc67-00c64142843d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:39.480993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370dc50-bd18-481c-bc67-00c64142843d None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:39.481383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370dc50-bd18-481c-bc67-00c64142843d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:39.482026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370dc50-bd18-481c-bc67-00c64142843d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:39.485181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370dc50-bd18-481c-bc67-00c64142843d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:39.509776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 418/857] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:39.509776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c623-13de-468c-8c6e-b5b6bf6f6be6 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:39.509776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c623-13de-468c-8c6e-b5b6bf6f6be6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:39.509776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9c623-13de-468c-8c6e-b5b6bf6f6be6 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:39.510519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 419/858] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:39.516518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c623-13de-468c-8c6e-b5b6bf6f6be6 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:39.619871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:39.662697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a96b4-f1db-4a74-a77f-e8774878077f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:39.688573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9c623-13de-468c-8c6e-b5b6bf6f6be6 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:39.702439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:39.714995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a4db-b293-40fb-9b12-9d7e12785fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:39.717154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a4db-b293-40fb-9b12-9d7e12785fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:39.717154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a4db-b293-40fb-9b12-9d7e12785fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:39.717620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a4db-b293-40fb-9b12-9d7e12785fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:39.737743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a4db-b293-40fb-9b12-9d7e12785fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:39.739360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a4db-b293-40fb-9b12-9d7e12785fe6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8b40a4db-b293-40fb-9b12-9d7e12785fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:39.761095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:39.762347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:39.763258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:39.763258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:39.763258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:39.763450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:39.763450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:39.763450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:39.763450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:39.768838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 440/859] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:39.770756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bd3fb-4a7e-490a-a512-8c83aa794b98 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:39.771077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bd3fb-4a7e-490a-a512-8c83aa794b98 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:39.771413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bd3fb-4a7e-490a-a512-8c83aa794b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:39.777210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:39.780781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:39.799629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:39.800615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:39.801632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 420/860] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:39.802703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:39.804736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-174bd3fb-4a7e-490a-a512-8c83aa794b98 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:39.806559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-174bd3fb-4a7e-490a-a512-8c83aa794b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:39.901500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 441/861] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:39.901996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bd3fb-4a7e-490a-a512-8c83aa794b98 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:39.965145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6bf03-84c0-4a40-91b5-3532af0ca09b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.006082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:40.012954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.013619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.013958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.014544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.015529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:40.015844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:40.017255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:40.031453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 421/862] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:40.042524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:40.090099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bd3fb-4a7e-490a-a512-8c83aa794b98 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.111412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:40.152796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.153883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.154380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.155422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.156915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:40.157483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:40.160237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:40.180699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 442/863] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:40.184360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a3d64-498d-4b0b-a1f4-b952a180d0d0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.189869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:40.230683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.231671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.232228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.232501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.242215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:40.244237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:40.262110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:40.263307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:40.265071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:40.265071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:40.265071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:40.265071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:40.265071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:40.265071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:40.265071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:40.265071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:40.266595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 422/864] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:40.267606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:40.321542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b30c5-14c5-4322-a542-03d2e7b01ecb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.351817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:40.375244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.376229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.376690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.377152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.378121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=4ba4c42b47794c9ba5fc6ee557e2cfc3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:40.379418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:40.381985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:40.401750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 443/865] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:40 2022] GET /identity/v3/role_assignments?user.id=4ba4c42b47794c9ba5fc6ee557e2cfc3&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:40.403848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:40.411774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bfa59-c645-4c28-bc2c-cb285c2c51ca None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.458122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.458907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.459268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.459618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.460335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:40.461118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:40.463922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:40.483346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:40.484971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:40.487357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:40.490498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 423/866] 10.222.0.58 () {56 vars in 1132 bytes} [Mon Oct 3 13:17:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:40.491876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:40.585350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba16da-037a-412d-9d2c-a810eb4a7996 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.633492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.634145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.634430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.634710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.635438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:40.635746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:40.637249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:40.650784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3696c-d62b-49b7-989c-4f842aca7f9d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.666600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 444/867] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:40.675611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:40.677770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:40.708613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.709273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.709809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.710135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.710806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:40.711124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:40.713589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:40.721255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 424/868] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:40.732554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:40.894084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:40.894699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49992-f3c0-44ae-b9fa-5cf173bc9d6a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.911892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f88a21-accf-4a2f-8ff1-a1acb5a90447 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:40.935409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5ab54-5d6e-4ba2-8f5d-7bf58ab558f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.936213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5ab54-5d6e-4ba2-8f5d-7bf58ab558f6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.936609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5ab54-5d6e-4ba2-8f5d-7bf58ab558f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.936957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5ab54-5d6e-4ba2-8f5d-7bf58ab558f6 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.937756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5ab54-5d6e-4ba2-8f5d-7bf58ab558f6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:40.938138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5ab54-5d6e-4ba2-8f5d-7bf58ab558f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:40.939670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5ab54-5d6e-4ba2-8f5d-7bf58ab558f6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:40.949605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 425/869] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:40.991776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd75dc-c913-49fb-9bbf-23caef0e88f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:40.993168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd75dc-c913-49fb-9bbf-23caef0e88f4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:40.993950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd75dc-c913-49fb-9bbf-23caef0e88f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:40.994642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd75dc-c913-49fb-9bbf-23caef0e88f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:40.996244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd75dc-c913-49fb-9bbf-23caef0e88f4 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c1ff08df377e4a83aae3afedd204e1ef) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:40.997042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd75dc-c913-49fb-9bbf-23caef0e88f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:41.000034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd75dc-c913-49fb-9bbf-23caef0e88f4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:41.013248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 445/870] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:40 2022] GET /identity/v3/role_assignments?user.id=c1ff08df377e4a83aae3afedd204e1ef&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 608 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:41.962067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc36b4-9ec5-4f5d-ad30-c1eb9e8f3cf2 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:41.965912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc36b4-9ec5-4f5d-ad30-c1eb9e8f3cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:41.965912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc36b4-9ec5-4f5d-ad30-c1eb9e8f3cf2 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:41.965912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 426/871] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:41.973336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b6c0c-2aea-4435-84b8-9662d974f015 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:41.974330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b6c0c-2aea-4435-84b8-9662d974f015 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:41.974911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b6c0c-2aea-4435-84b8-9662d974f015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:42.000527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-342b6c0c-2aea-4435-84b8-9662d974f015 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:42.003056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-342b6c0c-2aea-4435-84b8-9662d974f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:42.092865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 446/872] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:42.383151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc36b4-9ec5-4f5d-ad30-c1eb9e8f3cf2 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:42.458941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:42.551280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc36b4-9ec5-4f5d-ad30-c1eb9e8f3cf2 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:42.598907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9c911-7402-4926-9b69-273dae4fe937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:42.599455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c911-7402-4926-9b69-273dae4fe937 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:42.599741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c911-7402-4926-9b69-273dae4fe937 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:42.600018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9c911-7402-4926-9b69-273dae4fe937 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:42.600695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9c911-7402-4926-9b69-273dae4fe937 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:42.602125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9c911-7402-4926-9b69-273dae4fe937 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:42.621773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 427/873] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:17:42 2022] POST /identity/v3/services => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:43.361305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8a47c-7a98-436b-a2a3-ff1426ffd0ad None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:43.362205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8a47c-7a98-436b-a2a3-ff1426ffd0ad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:43.362634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8a47c-7a98-436b-a2a3-ff1426ffd0ad None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:43.364406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 447/874] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:43.370328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7868-d582-4f34-a596-1327f2489824 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:43.371217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7868-d582-4f34-a596-1327f2489824 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:43.371711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7868-d582-4f34-a596-1327f2489824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:43.385723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7ed3-fe1e-4f8e-9409-9983537f7908 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:43.386374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7ed3-fe1e-4f8e-9409-9983537f7908 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:43.386876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb7ed3-fe1e-4f8e-9409-9983537f7908 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:43.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 448/875] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:17:43.395311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575693a2-98a9-4660-a5f8-2659d0334607 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:43.395831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575693a2-98a9-4660-a5f8-2659d0334607 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:43.396319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575693a2-98a9-4660-a5f8-2659d0334607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:43.401487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-248b7868-d582-4f34-a596-1327f2489824 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:43.402926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-248b7868-d582-4f34-a596-1327f2489824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:43.421362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-575693a2-98a9-4660-a5f8-2659d0334607 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:43.423198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-575693a2-98a9-4660-a5f8-2659d0334607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:43.493479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 428/876] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:43.522693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 449/877] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:43.720059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a8de-9b53-4913-b7dd-905969daba3c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:43.720425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a8de-9b53-4913-b7dd-905969daba3c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:43.720615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964a8de-9b53-4913-b7dd-905969daba3c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:43.722411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 429/878] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:43.728314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d78e3-777f-486c-8903-23689efdd18a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:43.728606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d78e3-777f-486c-8903-23689efdd18a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:43.728870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d78e3-777f-486c-8903-23689efdd18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:43.748290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c13d78e3-777f-486c-8903-23689efdd18a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:43.749879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c13d78e3-777f-486c-8903-23689efdd18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:43.765336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964a8de-9b53-4913-b7dd-905969daba3c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:43.771524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:43.799067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:43.846021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 450/879] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:43.846750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d78e3-777f-486c-8903-23689efdd18a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:43.888242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964a8de-9b53-4913-b7dd-905969daba3c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:43.933810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:43.935055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:43.936143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:43.936670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:43.937505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:43.939358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:43.950096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:43.951523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:43.952616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:43.961597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 430/880] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:43.962949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:43.981656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d78e3-777f-486c-8903-23689efdd18a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:44.031751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659309-6672-49f4-80a2-42f955064109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:44.032859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659309-6672-49f4-80a2-42f955064109 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:44.033609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659309-6672-49f4-80a2-42f955064109 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:44.034204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57659309-6672-49f4-80a2-42f955064109 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:44.049548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57659309-6672-49f4-80a2-42f955064109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-57659309-6672-49f4-80a2-42f955064109 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:44.056249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:44.057695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:17:44.058733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:44.061840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 451/881] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Oct 3 13:17:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:44.073564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659309-6672-49f4-80a2-42f955064109 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:44.080215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:44.094408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:44.105697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cf26f-0a94-4ad9-81d3-d8ee774ed4e3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:44.156467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:44.157095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:44.157387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:44.157697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:44.158351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:44.158661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:44.160077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:44.164974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 431/882] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:17:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:44.165814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:44.215408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57659309-6672-49f4-80a2-42f955064109 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:44.273665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:44.274264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:44.274551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:44.274832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:44.275710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:44.276719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:44.278837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:44.290972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79509d1-091a-4482-b969-2753f979ae71 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:44.293831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 452/883] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Oct 3 13:17:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:44.294811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:44.348003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069d3d1-6aa1-403f-9677-247f15e00c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:44.349023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d3d1-6aa1-403f-9677-247f15e00c6b None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:44.349579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d3d1-6aa1-403f-9677-247f15e00c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:44.350057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069d3d1-6aa1-403f-9677-247f15e00c6b None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:44.350970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069d3d1-6aa1-403f-9677-247f15e00c6b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:44.352588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069d3d1-6aa1-403f-9677-247f15e00c6b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:44.369224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 432/884] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:17:44 2022] POST /identity/v3/services => generated 269 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:44.370546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069d3d1-6aa1-403f-9677-247f15e00c6b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:44.412814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:44.423747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:44.473866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ced035-6fed-406e-b665-99d31f9a9593 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:44.538535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:44.539460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:44.539808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:44.540155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:44.540911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:44.541311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:44.542920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:44.561496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 453/885] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:17:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:44.587323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069d3d1-6aa1-403f-9677-247f15e00c6b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:44.630584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:44.631259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:44.631540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:44.631814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:44.644852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:44.646418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:44.663804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:44.665879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:44.667707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:44.667707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:44.667707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:44.667707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 433/886] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:44.678645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:44.882980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597f829-c107-4151-8edb-fc54073061d6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:44.943451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:44.954187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5ff41-fa4a-4cbf-9570-4aee9c564bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:44.955240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ff41-fa4a-4cbf-9570-4aee9c564bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:44.955738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ff41-fa4a-4cbf-9570-4aee9c564bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:44.956233 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5ff41-fa4a-4cbf-9570-4aee9c564bc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:44.957635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5ff41-fa4a-4cbf-9570-4aee9c564bc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:44.958247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5ff41-fa4a-4cbf-9570-4aee9c564bc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:44.960936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5ff41-fa4a-4cbf-9570-4aee9c564bc1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:44.979366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 454/887] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:44.988919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:45.117540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:45.162591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909d6a-698d-4a3b-b0fd-22b24ab0b3f4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:45.203944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:45.204981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:45.205601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:45.206105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:45.218915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:45.220456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:45.225883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6d7f-67a3-4706-a9d1-a54b13d75cc6 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:45.226502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6d7f-67a3-4706-a9d1-a54b13d75cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:45.226983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead6d7f-67a3-4706-a9d1-a54b13d75cc6 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:45.230023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 455/888] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:45.234588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:45.235609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:45.236983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 434/889] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:17:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:45.238787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3250-2bf5-4de2-a486-38e9ed9c9270 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:45.239283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3250-2bf5-4de2-a486-38e9ed9c9270 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:45.239759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3250-2bf5-4de2-a486-38e9ed9c9270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:45.244563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:45.260744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-164a3250-2bf5-4de2-a486-38e9ed9c9270 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:45.262030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-164a3250-2bf5-4de2-a486-38e9ed9c9270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:45.361607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 456/890] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:45.442391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cfc2e-0c4e-4689-ba6d-4d8599d839ef None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:45.480810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:45.488298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:45.488927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:45.489211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:45.489496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:45.490244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:45.490725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:45.492184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:45.517139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 435/891] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Oct 3 13:17:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:45.526677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a3250-2bf5-4de2-a486-38e9ed9c9270 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:45.647056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:45.724541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a3250-2bf5-4de2-a486-38e9ed9c9270 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:45.756001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:45.784388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabd765-115c-4b44-a435-aa4ebcce9349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:45.785084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabd765-115c-4b44-a435-aa4ebcce9349 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:45.785572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabd765-115c-4b44-a435-aa4ebcce9349 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:45.786053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabd765-115c-4b44-a435-aa4ebcce9349 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:45.810210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32165a22-b25c-437a-849b-f2515803f449 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:45.859338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:45.860000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:45.860289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:45.860569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:45.861168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:45.863302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:45.876402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:45.877487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:45.878349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:45.881580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 436/892] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:45.886299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:45.917698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabd765-115c-4b44-a435-aa4ebcce9349 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:45.921081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabd765-115c-4b44-a435-aa4ebcce9349 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:45.963093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:46.037862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3557a37-ecd5-451a-9b86-bd3d9539959a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:46.078431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 457/893] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:45 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:17:46.084084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabd765-115c-4b44-a435-aa4ebcce9349 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:46.088936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:46.089889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:46.090373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:46.090842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:46.091677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:46.092821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:46.094436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:46.100584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 437/894] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:46.106334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:46.120922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:46.248851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca715111-c768-4082-b2f7-f9ed48521a70 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:46.249850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabd765-115c-4b44-a435-aa4ebcce9349 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:46.268123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:46.288584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:46.289111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:46.289389 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:46.289684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:46.290326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:46.290629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:46.292040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:46.298361 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 438/895] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:46.307801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:46.322470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:46.323167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:46.323440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:46.323722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:46.343603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:46.348297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:46.383902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:46.385416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:46.387048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:46.387048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:46.387048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:46.391723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 458/896] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:46.402989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:46.479716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c320604-b1d7-4a6a-a645-6b16685ecdb4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:46.500613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:46.543136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f6192-a7a2-46e8-ab1d-0a15b03abef6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:46.548511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:46.549714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:46.550354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:46.551006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:46.552318 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] RBAC: Adding query filter params (interface=public, service_id=66c527dfbc7941f49478fd18bb4adac1, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:46.552997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:46.556088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:46.578733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 439/897] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Oct 3 13:17:46 2022] GET /identity/v3/endpoints?service_id=66c527dfbc7941f49478fd18bb4adac1&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:46.583822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:46.597506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:46.599219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:46.599717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:46.600172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:46.601541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:46.602083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:46.604408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:46.622209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 459/898] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:46.628742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:46.655587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:46.729071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a48f3-48f6-44b0-9425-bf5ee15864d7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:46.749292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f5bc4-d640-4d4c-9369-bfcbbcdcc6dd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:46.773703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301c41b-4d3d-47f8-a713-ecb48579398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:46.774804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301c41b-4d3d-47f8-a713-ecb48579398c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:46.775288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301c41b-4d3d-47f8-a713-ecb48579398c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:46.775761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301c41b-4d3d-47f8-a713-ecb48579398c None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:46.776586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301c41b-4d3d-47f8-a713-ecb48579398c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:46.778565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301c41b-4d3d-47f8-a713-ecb48579398c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:46.785702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:46.794009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 440/899] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:17:46 2022] GET /identity/v3/services => generated 1268 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:17:46.798252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217fb0c-5930-4553-8281-555aea9de511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:46.799436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fb0c-5930-4553-8281-555aea9de511 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:46.799939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fb0c-5930-4553-8281-555aea9de511 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:46.800446 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217fb0c-5930-4553-8281-555aea9de511 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:46.813000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217fb0c-5930-4553-8281-555aea9de511 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:46.814795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217fb0c-5930-4553-8281-555aea9de511 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c217fb0c-5930-4553-8281-555aea9de511 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:46.834499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:46.835475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:46.836370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 460/900] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:46.844193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301c41b-4d3d-47f8-a713-ecb48579398c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:47.021247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301c41b-4d3d-47f8-a713-ecb48579398c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:47.039683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:47.057670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e781b1-68ce-4724-ad94-83347f8ab7bf None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.058299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e781b1-68ce-4724-ad94-83347f8ab7bf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.058781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e781b1-68ce-4724-ad94-83347f8ab7bf None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.060416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 461/901] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:47.067465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cb947-bf95-4297-b5b0-9581dae8cba2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.068367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cb947-bf95-4297-b5b0-9581dae8cba2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.068862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cb947-bf95-4297-b5b0-9581dae8cba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.088027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:47.088742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.089023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.089298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.090049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:47.090363 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:47.093252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:47.094477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3b2cb947-bf95-4297-b5b0-9581dae8cba2 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:47.096304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3b2cb947-bf95-4297-b5b0-9581dae8cba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:47.109000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 441/902] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:47.117286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:47.213272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 462/903] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:47.284806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8631660-117a-471c-b1bc-263b54632246 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.285325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8631660-117a-471c-b1bc-263b54632246 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.285878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8631660-117a-471c-b1bc-263b54632246 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.291736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 463/904] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:47.297104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b728-9399-4a77-b7c1-d152dd213140 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.298420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b728-9399-4a77-b7c1-d152dd213140 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.299391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b728-9399-4a77-b7c1-d152dd213140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.319619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b77fd-8555-43cc-9b41-d9deff64a098 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:47.343169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a280b728-9399-4a77-b7c1-d152dd213140 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:47.344527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a280b728-9399-4a77-b7c1-d152dd213140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:47.386244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:47.386871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.387151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.387511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.389847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] RBAC: Adding query filter params (role.id=ebc793a410f84acbbe3e2ba7c6509a35, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c1ff08df377e4a83aae3afedd204e1ef) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:47.390962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:47.393756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:47.406273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 442/905] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:47 2022] GET /identity/v3/role_assignments?user.id=c1ff08df377e4a83aae3afedd204e1ef&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=ebc793a410f84acbbe3e2ba7c6509a35 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:47.472711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 464/906] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:47.492938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:47.549123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:47.633767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8760620-bba1-49ba-992d-8943549ffae4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:47.674585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:47.675566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.675932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.676206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.676818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:47.678253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:47.690142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:47.691201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:47.692134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:47.693521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 443/907] 10.222.0.58 () {56 vars in 1132 bytes} [Mon Oct 3 13:17:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:47.699721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b728-9399-4a77-b7c1-d152dd213140 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:47.783305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:47.791381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:47.866784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b728-9399-4a77-b7c1-d152dd213140 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:47.937366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329559fc-5a22-4d90-a287-190af4bb22ef None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:47.939542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:47.940482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.940773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.941104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.942217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:47.942721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:47.944555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:47.953086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 465/908] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:47.954004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:47.959590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:47.989082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76806f9-1239-4a1b-8a0d-906412a0f12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:47.989743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76806f9-1239-4a1b-8a0d-906412a0f12e None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:47.990027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76806f9-1239-4a1b-8a0d-906412a0f12e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:47.990306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76806f9-1239-4a1b-8a0d-906412a0f12e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:47.990989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76806f9-1239-4a1b-8a0d-906412a0f12e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:47.993229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76806f9-1239-4a1b-8a0d-906412a0f12e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:48.015570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 444/909] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:17:47 2022] POST /identity/v3/services => generated 241 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:17:48.094405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bd197-8a50-4ee9-9324-703449d8f5e5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:48.116334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:48.146045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:48.147066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:48.147652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:48.148151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:48.149041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:48.149591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:48.151384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:48.158584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 466/910] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:48.165962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76806f9-1239-4a1b-8a0d-906412a0f12e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:48.283792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76806f9-1239-4a1b-8a0d-906412a0f12e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:48.321797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5ef3a-93b6-4501-aaaf-093e9a6b90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:48.322484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5ef3a-93b6-4501-aaaf-093e9a6b90ef None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:48.322775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5ef3a-93b6-4501-aaaf-093e9a6b90ef None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:48.323053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5ef3a-93b6-4501-aaaf-093e9a6b90ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:48.323773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5ef3a-93b6-4501-aaaf-093e9a6b90ef None admin] RBAC: Adding query filter params (interface=public, service_id=c119f2a845f94fe5af25bf534cb177c3, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:48.324089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5ef3a-93b6-4501-aaaf-093e9a6b90ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:48.325498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5ef3a-93b6-4501-aaaf-093e9a6b90ef None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:48.332514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 445/911] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Oct 3 13:17:48 2022] GET /identity/v3/endpoints?service_id=c119f2a845f94fe5af25bf534cb177c3&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:48.338371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:48.493748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17c54-c13f-4737-a3a4-2490eae83ef8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:48.554819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dce5a-1014-4bd6-8065-193a22c14c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:48.555915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dce5a-1014-4bd6-8065-193a22c14c59 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:48.556448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dce5a-1014-4bd6-8065-193a22c14c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:48.556956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dce5a-1014-4bd6-8065-193a22c14c59 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:48.558012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dce5a-1014-4bd6-8065-193a22c14c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:48.560237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dce5a-1014-4bd6-8065-193a22c14c59 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:48.573204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 467/912] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:17:48 2022] GET /identity/v3/services => generated 1497 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:17:49.733488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da26f-4906-4d40-a549-f2a9795ed02e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:49.734354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da26f-4906-4d40-a549-f2a9795ed02e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:49.734354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55da26f-4906-4d40-a549-f2a9795ed02e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:49.735750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 446/913] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:49.749499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097878-bd1c-45f3-a0ce-2bd17b8561f9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:49.749499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097878-bd1c-45f3-a0ce-2bd17b8561f9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:49.749499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097878-bd1c-45f3-a0ce-2bd17b8561f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:49.783369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ca097878-bd1c-45f3-a0ce-2bd17b8561f9 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:49.786065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ca097878-bd1c-45f3-a0ce-2bd17b8561f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:49.857947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:49.871009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611558a-8cbf-43ca-8c94-f817d432eb3f None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:49.872023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611558a-8cbf-43ca-8c94-f817d432eb3f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:49.872845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611558a-8cbf-43ca-8c94-f817d432eb3f None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:49.876861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 447/914] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:49.886956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695369-097d-4fae-ab6e-7588b3cc1274 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:49.887895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695369-097d-4fae-ab6e-7588b3cc1274 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:49.888745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695369-097d-4fae-ab6e-7588b3cc1274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:49.917299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f7695369-097d-4fae-ab6e-7588b3cc1274 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:49.917299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f7695369-097d-4fae-ab6e-7588b3cc1274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:49.918373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 468/915] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:49.933845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:50.013165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 448/916] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:50.239365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca097878-bd1c-45f3-a0ce-2bd17b8561f9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:50.373723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca097878-bd1c-45f3-a0ce-2bd17b8561f9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:50.412256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:50.412974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:50.413266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:50.413575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:50.414184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:50.415597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:50.425402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:50.426736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:50.427991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 469/917] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:50.432864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7695369-097d-4fae-ab6e-7588b3cc1274 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:50.471258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:50.511983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:50.585684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7695369-097d-4fae-ab6e-7588b3cc1274 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:50.621422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568d428-82e1-48ce-95ce-5da0ee32bd92 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:50.657189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:50.658299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:50.658987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:50.659269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:50.661116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:50.661718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:50.665741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:50.681503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 449/918] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:50.686014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:50.690018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0fe06-21c3-4799-ae54-2caffe7e3f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:50.692266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0fe06-21c3-4799-ae54-2caffe7e3f9f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:50.692811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0fe06-21c3-4799-ae54-2caffe7e3f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:50.693274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0fe06-21c3-4799-ae54-2caffe7e3f9f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:50.710717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c0fe06-21c3-4799-ae54-2caffe7e3f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:50.714672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-94c0fe06-21c3-4799-ae54-2caffe7e3f9f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:50.722413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:50.724036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:17:50.725650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:50.730312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 470/919] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Oct 3 13:17:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:50.740143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0fe06-21c3-4799-ae54-2caffe7e3f9f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:50.855855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd43b33-5977-4cfb-b826-2947b9d046cc None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:50.901431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:50.913813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0fe06-21c3-4799-ae54-2caffe7e3f9f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:50.921719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422d5ec-6c9f-4065-b545-f4a72e2d4403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:50.922488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422d5ec-6c9f-4065-b545-f4a72e2d4403 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:50.922824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422d5ec-6c9f-4065-b545-f4a72e2d4403 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:50.923144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422d5ec-6c9f-4065-b545-f4a72e2d4403 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:50.923914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422d5ec-6c9f-4065-b545-f4a72e2d4403 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:50.924222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422d5ec-6c9f-4065-b545-f4a72e2d4403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:50.925826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422d5ec-6c9f-4065-b545-f4a72e2d4403 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:50.933602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 450/920] 10.222.0.58 () {56 vars in 1114 bytes} [Mon Oct 3 13:17:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:50.934540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd5ef2-70db-49ca-89c1-cde8c077dd5e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:50.934835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd5ef2-70db-49ca-89c1-cde8c077dd5e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:50.935118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd5ef2-70db-49ca-89c1-cde8c077dd5e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:50.941531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 451/921] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:50.942453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd5ef2-70db-49ca-89c1-cde8c077dd5e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:50.968121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b9611-225f-46c5-8fc0-163c37e1d718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:50.969076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b9611-225f-46c5-8fc0-163c37e1d718 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:50.969586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b9611-225f-46c5-8fc0-163c37e1d718 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:50.970095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b9611-225f-46c5-8fc0-163c37e1d718 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:50.970881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b9611-225f-46c5-8fc0-163c37e1d718 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:50.971189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b9611-225f-46c5-8fc0-163c37e1d718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:50.972630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19b9611-225f-46c5-8fc0-163c37e1d718 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:50.983330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 471/922] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Oct 3 13:17:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:50.989751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb39e2-8b7b-4084-8fbb-3cee3ab77808 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:50.990036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb39e2-8b7b-4084-8fbb-3cee3ab77808 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:50.990324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb39e2-8b7b-4084-8fbb-3cee3ab77808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.010483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-57cb39e2-8b7b-4084-8fbb-3cee3ab77808 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:51.012005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-57cb39e2-8b7b-4084-8fbb-3cee3ab77808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:51.046996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:51.075152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd5ef2-70db-49ca-89c1-cde8c077dd5e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:51.094750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 472/923] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:51.095639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb39e2-8b7b-4084-8fbb-3cee3ab77808 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:51.136353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36707db9-4a40-4141-9279-036968133261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:51.136954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707db9-4a40-4141-9279-036968133261 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:51.137238 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707db9-4a40-4141-9279-036968133261 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:51.137608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707db9-4a40-4141-9279-036968133261 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.138276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36707db9-4a40-4141-9279-036968133261 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:51.140737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36707db9-4a40-4141-9279-036968133261 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:51.173161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 452/924] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:17:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:51.174930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3acdb7-fd79-423e-8799-7952ff2ddea8 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:51.175270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3acdb7-fd79-423e-8799-7952ff2ddea8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:51.175523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3acdb7-fd79-423e-8799-7952ff2ddea8 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.177098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 453/925] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:51.183410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc827f3f-ef7e-4eaf-a0a3-c97a531ac751 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:51.183647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc827f3f-ef7e-4eaf-a0a3-c97a531ac751 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:51.183854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc827f3f-ef7e-4eaf-a0a3-c97a531ac751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.209091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-dc827f3f-ef7e-4eaf-a0a3-c97a531ac751 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:51.209091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dc827f3f-ef7e-4eaf-a0a3-c97a531ac751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:51.250453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:51.277254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb39e2-8b7b-4084-8fbb-3cee3ab77808 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:51.372379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:51.375290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:51.376160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:51.377429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.385493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 454/926] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:51.399538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:51.401254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc827f3f-ef7e-4eaf-a0a3-c97a531ac751 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:51.403116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:51.423525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:51.424501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:51.425399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:51.425399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:51.425399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:51.425399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 473/927] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:51.433605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:51.489120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:51.561275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc827f3f-ef7e-4eaf-a0a3-c97a531ac751 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:51.578482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a75fdb-9352-47aa-b089-5bb7ac8f4d11 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:51.626209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:51.626879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:51.627157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:51.627417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.628089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:51.628397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:51.629925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:51.632324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:51.635664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:51.637424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:51.638284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:51.640695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.642491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:51.652570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:51.652835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 474/928] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:51.656104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:51.660531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:51.662172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:51.663541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 455/929] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:51.667944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:51.798309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44980bd-572a-489e-80d2-702d7c0decd2 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:51.800974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9897873a-22ef-43ec-a68f-5ad13cef8c8d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:51.830029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:51.848801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:51.849832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:51.850590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:51.851103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.852535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:51.853387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:51.855297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:51.869487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 456/930] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:17:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:51.874929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:51.875988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:51.876657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:51.876943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:51.877210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:51.887322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:51.889726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:51.905097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:51.906746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:51.907767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:51.911280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 475/931] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:17:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:51.912442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:51.977881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:52.046365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a25fd3-24e6-4a31-b2e3-ecdd5734cb9f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.053111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b0388-1d02-4506-a8b8-883fd6c4c391 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.102172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.102813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.103128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.103584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.104666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:52.105171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.108173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.110434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.111388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.111791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.112252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.113508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:52.113997 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.116445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.121672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 457/932] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:17:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:52.121672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.149538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 476/933] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Oct 3 13:17:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:52.151030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.206925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:52.273006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b440189-f956-4718-8448-6397b1c79a0d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.284977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db89f29-9aa7-49d8-ab40-888101700959 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.301509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:52.317102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.317706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.318026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.318306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.319036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.320431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.335700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.335700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.335700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.335700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.336300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] RBAC: Adding query filter params (interface=public, service_id=a56f951fa76546ddabafebaf6e26c1a8, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:52.336451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:52.338586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 03 13:17:52.340352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:52.344981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.345647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 458/934] 10.222.0.58 () {56 vars in 1132 bytes} [Mon Oct 3 13:17:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:52.345647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.346048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.356875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 477/935] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Oct 3 13:17:52 2022] GET /identity/v3/endpoints?service_id=a56f951fa76546ddabafebaf6e26c1a8&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:52.358271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.492800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9964bc4-1638-4c0f-aa71-4da3f2c940ec None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.508333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5480f-da01-4475-ac1a-2a6e6bfd258c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.527454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:52.539758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.540605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.540977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.541341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.542348 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:52.542753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.544389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.551624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 478/936] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:52.555871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.562788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.563437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.563717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.563991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.576952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.578481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:52.593026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:52.594236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:52.595330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:52.595330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:52.595330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:52.601614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 459/937] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:52.601614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.617225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:52.683630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ed08f-4b80-4559-92d9-89e479b0b5d8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.710237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f71d41-8ecb-4479-b564-042ca5f2a53b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.722664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e5f22-930f-4be4-a3a4-20ae27286151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.723297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e5f22-930f-4be4-a3a4-20ae27286151 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.723578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e5f22-930f-4be4-a3a4-20ae27286151 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.723854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e5f22-930f-4be4-a3a4-20ae27286151 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.724470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e5f22-930f-4be4-a3a4-20ae27286151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.725899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e5f22-930f-4be4-a3a4-20ae27286151 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.741498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 479/938] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:17:52 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:17:52.741498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e5f22-930f-4be4-a3a4-20ae27286151 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.766293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.766935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.767220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.767499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.768191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:52.768503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.769995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.776237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 460/939] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:52.782630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.844690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:52.887057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e5f22-930f-4be4-a3a4-20ae27286151 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.890889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:52.919390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec8a52-9de5-4771-b5db-2de2e0aab5b6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:52.934120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.935787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.936550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.937489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.937660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:52.938011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.939431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.953106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 480/940] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:52.961923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:52.964372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b47fc-affc-4a51-add6-0b9f45a5492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:52.965194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b47fc-affc-4a51-add6-0b9f45a5492e None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:52.965969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b47fc-affc-4a51-add6-0b9f45a5492e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:52.965969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b47fc-affc-4a51-add6-0b9f45a5492e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:52.966967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b47fc-affc-4a51-add6-0b9f45a5492e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:52.968616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b47fc-affc-4a51-add6-0b9f45a5492e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:52.994957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 461/941] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:17:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:17:53.105957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bad5ec-1626-4913-846e-1e00cb41ef36 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:53.156695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:53.157364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:53.157687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:53.157969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:53.166082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:53.167579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:53.182869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:53.183868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:53.184801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 481/942] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:53.192198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b47fc-affc-4a51-add6-0b9f45a5492e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:53.304548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:53.329162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b47fc-affc-4a51-add6-0b9f45a5492e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:53.370759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b84ea-87bb-411e-b94f-30e49309b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:53.371312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b84ea-87bb-411e-b94f-30e49309b712 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:53.371597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b84ea-87bb-411e-b94f-30e49309b712 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:53.371872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b84ea-87bb-411e-b94f-30e49309b712 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:53.372578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b84ea-87bb-411e-b94f-30e49309b712 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:53.373011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b84ea-87bb-411e-b94f-30e49309b712 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:53.374980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b84ea-87bb-411e-b94f-30e49309b712 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:53.390377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 462/943] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:53.396799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:53.425919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:53.527885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79bcbc-6908-49ba-9039-8d91bbdf2a6a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:53.569355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138c024-bfdd-4544-b487-aaf96ad65a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:53.570247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138c024-bfdd-4544-b487-aaf96ad65a88 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:53.570585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138c024-bfdd-4544-b487-aaf96ad65a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:53.570872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138c024-bfdd-4544-b487-aaf96ad65a88 None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c1ff08df377e4a83aae3afedd204e1ef/roles/ebc793a410f84acbbe3e2ba7c6509a35` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:53.573808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138c024-bfdd-4544-b487-aaf96ad65a88 None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=c1ff08df377e4a83aae3afedd204e1ef, role_id=ebc793a410f84acbbe3e2ba7c6509a35)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:53.576378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138c024-bfdd-4544-b487-aaf96ad65a88 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:53.605274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 482/944] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:17:53 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c1ff08df377e4a83aae3afedd204e1ef/roles/ebc793a410f84acbbe3e2ba7c6509a35 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:17:53.803058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df506-9b0a-4d7f-bfc2-67a9ad535b3b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:53.803646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df506-9b0a-4d7f-bfc2-67a9ad535b3b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:53.804090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38df506-9b0a-4d7f-bfc2-67a9ad535b3b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:53.806073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 463/945] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:53.812415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efd4b7-605a-4c31-9d2c-6eef00f5402f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:53.812954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efd4b7-605a-4c31-9d2c-6eef00f5402f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:53.813420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efd4b7-605a-4c31-9d2c-6eef00f5402f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:53.837188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-96efd4b7-605a-4c31-9d2c-6eef00f5402f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:53.838719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-96efd4b7-605a-4c31-9d2c-6eef00f5402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:53.940768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 483/946] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:54.228519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df506-9b0a-4d7f-bfc2-67a9ad535b3b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:54.353295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38df506-9b0a-4d7f-bfc2-67a9ad535b3b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:54.403308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:54.404484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:54.404813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:54.405297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:54.406200 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:54.407706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:54.410392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING keystone.server.flask.common [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 03 13:17:54.430540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.sql.core [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 13:17:54.430540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 03 13:17:54.430540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [parameters: {'id': 'd141ebd3f4c44fdfada2b52c0899d681', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 03 13:17:54.430540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81423) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d141ebd3f4c44fdfada2b52c0899d681.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d141ebd3f4c44fdfada2b52c0899d681. Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application conn.query(q) Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 13:17:54.440174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result.read() Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 03 13:17:54.441062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application next(self.gen) Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application yield resource Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application next(self.gen) Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise self.value Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application session.commit() Oct 03 13:17:54.442778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.session.flush() Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application compat.raise_( Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 13:17:54.444862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 13:17:54.449860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application conn.query(q) Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application result.read() Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 13:17:54.452260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [parameters: {'id': 'd141ebd3f4c44fdfada2b52c0899d681', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:54.454484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:54.455600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d141ebd3f4c44fdfada2b52c0899d681. Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:54.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 464/947] 10.222.0.58 () {60 vars in 1140 bytes} [Mon Oct 3 13:17:54 2022] POST /identity/v3/projects => generated 262 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:54.457351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efd4b7-605a-4c31-9d2c-6eef00f5402f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:54.490750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:54.572321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efd4b7-605a-4c31-9d2c-6eef00f5402f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:54.611416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d59392-c495-4c81-ac3f-6e8ef0188cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:54.612067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59392-c495-4c81-ac3f-6e8ef0188cad None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:54.612355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59392-c495-4c81-ac3f-6e8ef0188cad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:54.612636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d59392-c495-4c81-ac3f-6e8ef0188cad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:54.620190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d59392-c495-4c81-ac3f-6e8ef0188cad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:54.621696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d59392-c495-4c81-ac3f-6e8ef0188cad None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-14d59392-c495-4c81-ac3f-6e8ef0188cad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:54.634490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:54.637397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:54.641776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 484/948] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:54.643223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:54.649674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:54.759958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd7ecd-de7e-414e-980f-c76c79beee33 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:54.798277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f04081-af46-4344-a111-9eeece0f5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:54.798911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04081-af46-4344-a111-9eeece0f5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:54.799197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04081-af46-4344-a111-9eeece0f5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:54.799479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f04081-af46-4344-a111-9eeece0f5fb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:54.800173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f04081-af46-4344-a111-9eeece0f5fb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:54.800482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f04081-af46-4344-a111-9eeece0f5fb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:54.802207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f04081-af46-4344-a111-9eeece0f5fb6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:54.825367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 465/949] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:55.211281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a607-03e0-4e03-b47a-5b89c6946393 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:55.212044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a607-03e0-4e03-b47a-5b89c6946393 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:55.212044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53a607-03e0-4e03-b47a-5b89c6946393 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:55.213651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 485/950] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:55.219997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1275-c84c-42e6-b741-f7328856278d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:55.220322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1275-c84c-42e6-b741-f7328856278d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:55.220562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea1275-c84c-42e6-b741-f7328856278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:55.241615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fbea1275-c84c-42e6-b741-f7328856278d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:55.243259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fbea1275-c84c-42e6-b741-f7328856278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:55.318581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:55.328657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 466/951] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:55.614699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53a607-03e0-4e03-b47a-5b89c6946393 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:55.677115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899df3d4-d2da-4da0-9920-d319327237e3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:55.677437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899df3d4-d2da-4da0-9920-d319327237e3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:55.677755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899df3d4-d2da-4da0-9920-d319327237e3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:55.679535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 467/952] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:17:55.685212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2769-099a-417e-8a81-7004d6657318 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:55.685831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2769-099a-417e-8a81-7004d6657318 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:55.686307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2769-099a-417e-8a81-7004d6657318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:55.696040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:55.709829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c82e2769-099a-417e-8a81-7004d6657318 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:55.711512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c82e2769-099a-417e-8a81-7004d6657318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:55.739459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53a607-03e0-4e03-b47a-5b89c6946393 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:55.788885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:55.789893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:55.790174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:55.790447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:55.791050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:55.792449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:55.805018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:55.806296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:55.807289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:55.809576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 468/953] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:55.810676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 486/954] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:55.816751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2769-099a-417e-8a81-7004d6657318 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:55.932756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2769-099a-417e-8a81-7004d6657318 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:55.961667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:55.975405 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:55.976277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:55.976715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:55.977163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:55.978040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:55.978551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:55.980376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:55.987110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 469/955] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:17:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:55.993103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:56.098881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:56.110325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a44693-08b7-46b5-8680-03ce7cc502fd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:56.158849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:56.159674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:56.160058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:56.160432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:56.161186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:56.161628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:56.163135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:56.166970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:56.171007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 487/956] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:17:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:56.177647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:56.262100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7e575-e027-4d7b-9dd3-385b94c9ee4f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:56.310995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da3f49-0138-4f59-9746-5491c596bd02 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:56.322198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc32c0-fe3a-4ce6-8a7d-4b084aa70525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:56.323201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc32c0-fe3a-4ce6-8a7d-4b084aa70525 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:56.323634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc32c0-fe3a-4ce6-8a7d-4b084aa70525 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:56.324066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc32c0-fe3a-4ce6-8a7d-4b084aa70525 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:56.325078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc32c0-fe3a-4ce6-8a7d-4b084aa70525 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:56.327579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc32c0-fe3a-4ce6-8a7d-4b084aa70525 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-72dc32c0-fe3a-4ce6-8a7d-4b084aa70525 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:56.343737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:56.345096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:56.346182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 470/957] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:56.347017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3bac-2467-48d8-965f-0bc01640ec6f None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:56.347017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3bac-2467-48d8-965f-0bc01640ec6f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:56.347147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f3bac-2467-48d8-965f-0bc01640ec6f None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:56.348909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 471/958] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:56.351889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f3bac-2467-48d8-965f-0bc01640ec6f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:56.366603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bb2de-4678-4638-a63f-2c9b17e5e285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:56.367246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb2de-4678-4638-a63f-2c9b17e5e285 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:56.367532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb2de-4678-4638-a63f-2c9b17e5e285 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:56.367814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb2de-4678-4638-a63f-2c9b17e5e285 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:56.368525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bb2de-4678-4638-a63f-2c9b17e5e285 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:56.370085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bb2de-4678-4638-a63f-2c9b17e5e285 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:56.397504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 488/959] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:17:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:17:56.397504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697482d-d458-4c6a-acfd-3ea993aa544c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:56.397504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697482d-d458-4c6a-acfd-3ea993aa544c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:56.397504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697482d-d458-4c6a-acfd-3ea993aa544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:56.429065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6697482d-d458-4c6a-acfd-3ea993aa544c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:56.431692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6697482d-d458-4c6a-acfd-3ea993aa544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:56.452435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:56.523912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f3bac-2467-48d8-965f-0bc01640ec6f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:56.524731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:56.557794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 489/960] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:56.579841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:56.580484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:56.580769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:56.581048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:56.581742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] RBAC: Adding query filter params (name=compute) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:56.582100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:56.583746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:56.597857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 472/961] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:56.601123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697482d-d458-4c6a-acfd-3ea993aa544c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:56.728577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697482d-d458-4c6a-acfd-3ea993aa544c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:56.768285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:56.769208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:56.769667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:56.770176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:56.770964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:56.771433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:56.773069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:56.779298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 490/962] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:17:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:56.837856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:56.965032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333136e0-62f1-4656-997b-9d173c87d6bb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:56.982880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:57.010811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847515b4-12d1-4561-97f8-ed38a31cee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:57.011635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847515b4-12d1-4561-97f8-ed38a31cee4e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:57.012027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847515b4-12d1-4561-97f8-ed38a31cee4e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:57.012395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847515b4-12d1-4561-97f8-ed38a31cee4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:57.028075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847515b4-12d1-4561-97f8-ed38a31cee4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-847515b4-12d1-4561-97f8-ed38a31cee4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:17:57.033966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:57.035135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:17:57.036208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:57.038229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 473/963] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Oct 3 13:17:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:57.043748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:57.097336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:57.151961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0798e-1f1b-4859-8b39-cbd3d62ebdfb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:57.191493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe176a-4e44-4aba-81a6-c2bd0752b2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:57.192109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe176a-4e44-4aba-81a6-c2bd0752b2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:57.192388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe176a-4e44-4aba-81a6-c2bd0752b2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:57.192663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe176a-4e44-4aba-81a6-c2bd0752b2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:57.193339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe176a-4e44-4aba-81a6-c2bd0752b2f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:57.193674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe176a-4e44-4aba-81a6-c2bd0752b2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:57.195130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe176a-4e44-4aba-81a6-c2bd0752b2f6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:57.205340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 491/964] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Oct 3 13:17:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:57.213737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847515b4-12d1-4561-97f8-ed38a31cee4e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:57.330104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847515b4-12d1-4561-97f8-ed38a31cee4e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:57.369928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:57.370572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:57.370914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:57.371190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:57.376830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eabf1-c79c-40d2-903d-48dfac6aee66 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:57.377812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eabf1-c79c-40d2-903d-48dfac6aee66 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:57.378307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eabf1-c79c-40d2-903d-48dfac6aee66 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:57.379881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 492/965] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:17:57.384314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:57.385847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:57.386181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4cd1a-a351-44b1-b4b5-078f8c92f74d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:57.386671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4cd1a-a351-44b1-b4b5-078f8c92f74d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:57.387150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4cd1a-a351-44b1-b4b5-078f8c92f74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:57.403147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:57.404788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:57.405756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:57.405756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:57.405756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:57.406364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 474/966] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:57.410780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:57.414433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7ad4cd1a-a351-44b1-b4b5-078f8c92f74d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:57.415995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7ad4cd1a-a351-44b1-b4b5-078f8c92f74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:57.440883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:57.521478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 493/967] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:57.573557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a0037-47bf-4216-8517-a6b421c385c7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:57.611849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:57.612844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:57.613348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:57.613843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:57.614685 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:57.615184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:57.616783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:57.631364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 475/968] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:57.637489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4cd1a-a351-44b1-b4b5-078f8c92f74d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:57.697213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:57.760229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4cd1a-a351-44b1-b4b5-078f8c92f74d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:57.797087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:57.798209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:57.798710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:57.799186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:57.807467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:57.809260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:57.822455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:57.823460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 03 13:17:57.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:57.827950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 494/969] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:17:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:57.834687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:57.873513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:57.960757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:57.984376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233c7fa-076e-4a72-a7c8-c43e9c9c46bc None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:58.023153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:58.030137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4ebcd-7653-48d4-a867-272bbfdd4cb6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:58.034548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:58.035310 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:58.035700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:58.036080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:58.036977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:58.037420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:58.039192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:58.065737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 476/970] 10.222.0.58 () {56 vars in 1149 bytes} [Mon Oct 3 13:17:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:58.068301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:58.077730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:58.079135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:58.079605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:58.080023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:58.080817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:58.082849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:58.086358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING keystone.server.flask.common [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 03 13:17:58.104630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.sql.core [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 13:17:58.104630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 03 13:17:58.104630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [parameters: {'user_id': '15d31a19abc7410ebe2cfc5e104542b1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 03 13:17:58.104630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81424) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 03 13:17:58.127474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 13:17:58.128672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application yield resource Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.rollback() Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.force_reraise() Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise self.value Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application session.commit() Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 03 13:17:58.129827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._prepare_impl() Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.flush() Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._flush(objects) Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application compat.raise_( Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application flush_context.execute() Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rec.execute(self) Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 03 13:17:58.131126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 03 13:17:58.132205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 03 13:17:58.133232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [parameters: {'user_id': '15d31a19abc7410ebe2cfc5e104542b1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:58.135025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 03 13:17:58.136932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 03 13:17:58.141289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 03 13:17:58.141289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 03 13:17:58.141289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 03 13:17:58.141289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 03 13:17:58.141289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 03 13:17:58.141289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:58.149437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 495/971] 10.222.0.58 () {60 vars in 1131 bytes} [Mon Oct 3 13:17:57 2022] POST /identity/v3/users => generated 160 bytes in 273 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:58.152016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:58.226106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bec918-c3bf-42e3-9b2c-8fa1d5b388cb None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:58.276683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3338723-d611-4cd2-8a69-fb71ef606385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:58.277339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338723-d611-4cd2-8a69-fb71ef606385 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:58.277697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338723-d611-4cd2-8a69-fb71ef606385 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:58.277979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3338723-d611-4cd2-8a69-fb71ef606385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:58.291413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bcf45-67b5-40d0-8c77-e7c9673778c0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:58.297072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:58.352147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:58.353296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:58.353826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:58.354411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:58.363919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:58.366131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:58.370975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3338723-d611-4cd2-8a69-fb71ef606385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:58.372579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3338723-d611-4cd2-8a69-fb71ef606385 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:17:58.387591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:58.389017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 03 13:17:58.390117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:58.397878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 496/972] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:17:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:58.399340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:58.434008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:58.508579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 477/973] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:17:58 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 13:17:58.509814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3338723-d611-4cd2-8a69-fb71ef606385 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:58.530865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84b5b7-637d-4759-887d-7dcb284c6368 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:58.567773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:58.580964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:58.582086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:58.582843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:58.583388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:58.584473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:58.585093 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:58.587045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:58.638845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 497/974] 10.222.0.58 () {56 vars in 1101 bytes} [Mon Oct 3 13:17:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:58.681765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3338723-d611-4cd2-8a69-fb71ef606385 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:58.742446 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd4496-59da-460c-b6bb-b989962f0f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:58.743124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd4496-59da-460c-b6bb-b989962f0f7c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:58.743415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd4496-59da-460c-b6bb-b989962f0f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:58.743709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd4496-59da-460c-b6bb-b989962f0f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:58.760418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd4496-59da-460c-b6bb-b989962f0f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:58.761995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd4496-59da-460c-b6bb-b989962f0f7c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-50cd4496-59da-460c-b6bb-b989962f0f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:17:58.783712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:58.785069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:17:58.785996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:17:58.785996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:17:58.785996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:17:58.785996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 478/975] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:58.790962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:58.920259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:58.945333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdb0c0-b9c5-4048-ba4e-85adcc235c2d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:58.988044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:58.988684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:58.988975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:58.989251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:58.989949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:58.990256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:58.991664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:59.003956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a470f-ebae-4d4c-8413-39660de432df None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.004244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a470f-ebae-4d4c-8413-39660de432df None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.004540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a470f-ebae-4d4c-8413-39660de432df None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.006488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 479/976] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:17:59.008790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 498/977] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:59.012336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564ba8-2c96-424c-8b07-b542eaa1b19f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.012752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564ba8-2c96-424c-8b07-b542eaa1b19f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.012897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564ba8-2c96-424c-8b07-b542eaa1b19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.015732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:59.037754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4f564ba8-2c96-424c-8b07-b542eaa1b19f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:59.039316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4f564ba8-2c96-424c-8b07-b542eaa1b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:59.132340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 480/978] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:17:59.160641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbef861-0f02-4e82-96c0-f5a0e8d6d6fd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:59.208334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7353e-6ebc-46b4-a0b6-8303951368d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:59.208993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7353e-6ebc-46b4-a0b6-8303951368d7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.209272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7353e-6ebc-46b4-a0b6-8303951368d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.209579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7353e-6ebc-46b4-a0b6-8303951368d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.218204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7353e-6ebc-46b4-a0b6-8303951368d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:59.219744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7353e-6ebc-46b4-a0b6-8303951368d7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:59.222879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1af7353e-6ebc-46b4-a0b6-8303951368d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:17:59.235767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:59.236921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:59.238083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:17:59.238083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:17:59.238083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:17:59.238083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:17:59.238083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:17:59.238083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:17:59.238083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:17:59.238083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:59.245878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 499/979] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:17:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:17:59.247493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f564ba8-2c96-424c-8b07-b542eaa1b19f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:59.276360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:59.318231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb23703-8e34-40d3-9df9-4cd1d5afa033 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.318735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb23703-8e34-40d3-9df9-4cd1d5afa033 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.319230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb23703-8e34-40d3-9df9-4cd1d5afa033 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.321833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 500/980] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:17:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:17:59.328226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ae60b-a72b-4a96-8820-551446659bda None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.328604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ae60b-a72b-4a96-8820-551446659bda None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.328792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ae60b-a72b-4a96-8820-551446659bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.350300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4c5ae60b-a72b-4a96-8820-551446659bda None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:17:59.353836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4c5ae60b-a72b-4a96-8820-551446659bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:17:59.402879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f564ba8-2c96-424c-8b07-b542eaa1b19f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:59.442660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 501/981] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:17:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:17:59.443571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ae60b-a72b-4a96-8820-551446659bda None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:59.468028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:59.468722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.468722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.469655 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.470324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:59.470694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:59.472213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:59.485798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 481/982] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:17:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:17:59.494957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:59.583086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ae60b-a72b-4a96-8820-551446659bda None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:59.617506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:59.624697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:17:59.628728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:59.631229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.632072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.632557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.633632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:59.635964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:59.643323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e4fc9-f06f-4c40-9dae-0163591ac0c7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:17:59.649709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:17:59.650791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:17:59.651660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:17:59.652459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 502/983] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:17:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:17:59.658807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:59.691767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:59.693909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.693909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.693909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.694388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] RBAC: Adding query filter params (role.id=ebc793a410f84acbbe3e2ba7c6509a35, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c1ff08df377e4a83aae3afedd204e1ef) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:59.694714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:59.696172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:59.713714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 482/984] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:17:59 2022] GET /identity/v3/role_assignments?user.id=c1ff08df377e4a83aae3afedd204e1ef&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=ebc793a410f84acbbe3e2ba7c6509a35 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:59.722573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:59.822496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31c0d8-f889-4e58-9477-9af5f94aaf91 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:59.867341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:59.867971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.868084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf939014-0d2c-438b-bf34-becc90cf0a3c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:17:59.868252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.868523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.869178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:17:59.869515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:59.870988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:59.882109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 503/985] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:17:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:17:59.883944 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:17:59.923850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20badb47-fc14-4ff7-af3a-dc28500e8e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:17:59.924475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20badb47-fc14-4ff7-af3a-dc28500e8e5e None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:17:59.924755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20badb47-fc14-4ff7-af3a-dc28500e8e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:17:59.925027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20badb47-fc14-4ff7-af3a-dc28500e8e5e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:17:59.925730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20badb47-fc14-4ff7-af3a-dc28500e8e5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:17:59.927919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20badb47-fc14-4ff7-af3a-dc28500e8e5e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:17:59.947229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 483/986] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:17:59 2022] POST /identity/v3/services => generated 242 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:17:59.976112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:00.045288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79012d-c9a9-4250-b98d-84cf5722e6eb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:00.109632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:00.110220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:00.110494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:00.110773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:00.111438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:00.111750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:00.113217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:00.124849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 504/987] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:17:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:00.125805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20badb47-fc14-4ff7-af3a-dc28500e8e5e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:00.141265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:00.266895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20badb47-fc14-4ff7-af3a-dc28500e8e5e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:00.316623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406d36d-a0f6-42cf-ac3d-51ce3624edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:00.317290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406d36d-a0f6-42cf-ac3d-51ce3624edd8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:00.317603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406d36d-a0f6-42cf-ac3d-51ce3624edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:00.317875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406d36d-a0f6-42cf-ac3d-51ce3624edd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:00.318601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406d36d-a0f6-42cf-ac3d-51ce3624edd8 None admin] RBAC: Adding query filter params (interface=admin, service_id=a56f951fa76546ddabafebaf6e26c1a8, region_id=RegionOne) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:00.318911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406d36d-a0f6-42cf-ac3d-51ce3624edd8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:00.320412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406d36d-a0f6-42cf-ac3d-51ce3624edd8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:00.327928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 484/988] 10.222.0.58 () {56 vars in 1249 bytes} [Mon Oct 3 13:18:00 2022] GET /identity/v3/endpoints?service_id=a56f951fa76546ddabafebaf6e26c1a8&interface=admin®ion_id=RegionOne => generated 193 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:00.335816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:00.476254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676176c-0212-41dd-a4b8-6b3a20255491 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:00.496680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:00.522286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf15214-c9b4-479f-add1-5ae7b20938c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:00.522828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf15214-c9b4-479f-add1-5ae7b20938c1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:00.523117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf15214-c9b4-479f-add1-5ae7b20938c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:00.523385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf15214-c9b4-479f-add1-5ae7b20938c1 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:00.524005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf15214-c9b4-479f-add1-5ae7b20938c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:00.525406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf15214-c9b4-479f-add1-5ae7b20938c1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:00.545537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 505/989] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:18:00 2022] GET /identity/v3/services => generated 1727 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:18:02.043444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f45fe-15af-49e0-9d49-7c7c9d6ee678 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.043976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f45fe-15af-49e0-9d49-7c7c9d6ee678 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.043976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f45fe-15af-49e0-9d49-7c7c9d6ee678 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.049507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 485/990] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:02.052121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da872d-c23e-4371-9614-de00f76dee5b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.052586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da872d-c23e-4371-9614-de00f76dee5b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.052868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da872d-c23e-4371-9614-de00f76dee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.072172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d3da872d-c23e-4371-9614-de00f76dee5b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:02.073393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d3da872d-c23e-4371-9614-de00f76dee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:02.152438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 506/991] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:02.306646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5a82-5d5b-4194-b0b5-c8974abcbaae None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.306900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5a82-5d5b-4194-b0b5-c8974abcbaae None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.307182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5a82-5d5b-4194-b0b5-c8974abcbaae None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.309505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 486/992] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:02.315171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cab192-18f4-44ea-a615-0adf47898a1b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.315517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cab192-18f4-44ea-a615-0adf47898a1b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.315741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cab192-18f4-44ea-a615-0adf47898a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.345884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-21cab192-18f4-44ea-a615-0adf47898a1b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:02.345884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-21cab192-18f4-44ea-a615-0adf47898a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:02.433117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 507/993] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:02.437304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5a82-5d5b-4194-b0b5-c8974abcbaae None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:02.526162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:02.563608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5a82-5d5b-4194-b0b5-c8974abcbaae None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:02.599460 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:02.600093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.600371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.600649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] PATH_INFO: `/v3/services/image` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.601239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:02.602644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:02.612133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:02.614365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:02.615323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:02.617612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 487/994] 10.222.0.58 () {56 vars in 1105 bytes} [Mon Oct 3 13:18:02 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:02.619182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cab192-18f4-44ea-a615-0adf47898a1b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:02.668101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:02.710354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:02.735209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cab192-18f4-44ea-a615-0adf47898a1b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:02.772596 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:02.773271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.773602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.773801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.774440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] RBAC: Adding query filter params (name=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:02.774757 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:02.776155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:02.781374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 508/995] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:02.787234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:02.826100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9db4ee-09ca-4287-8c14-10352a8fcb7c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:02.862262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93e9cb-5d64-4dfa-a5a9-4c299ddcad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:02.862903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e9cb-5d64-4dfa-a5a9-4c299ddcad8b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.863186 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e9cb-5d64-4dfa-a5a9-4c299ddcad8b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.863461 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e9cb-5d64-4dfa-a5a9-4c299ddcad8b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.864053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93e9cb-5d64-4dfa-a5a9-4c299ddcad8b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:02.865429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93e9cb-5d64-4dfa-a5a9-4c299ddcad8b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:02.868149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-6e93e9cb-5d64-4dfa-a5a9-4c299ddcad8b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:02.876831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:02.878281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:18:02.879393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:02.881638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 488/996] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:18:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:02.882968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f18a4b-f5d3-4829-95c3-2f888d064db9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.883625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f18a4b-f5d3-4829-95c3-2f888d064db9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.883903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f18a4b-f5d3-4829-95c3-2f888d064db9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.885194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 489/997] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:02.885927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f18a4b-f5d3-4829-95c3-2f888d064db9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:02.901112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec8475-ca03-40ea-9de4-1819bd9de510 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:02.908786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:02.938992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54c87f-5c2a-45ae-a358-1f54341fcc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:02.940052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c87f-5c2a-45ae-a358-1f54341fcc3b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.940563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c87f-5c2a-45ae-a358-1f54341fcc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.941071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c87f-5c2a-45ae-a358-1f54341fcc3b None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.942585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54c87f-5c2a-45ae-a358-1f54341fcc3b None admin] RBAC: Adding query filter params (type=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:02.942585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54c87f-5c2a-45ae-a358-1f54341fcc3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:02.943516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54c87f-5c2a-45ae-a358-1f54341fcc3b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:02.951123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 509/998] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:02.959004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14024-a02f-4895-8379-5a5788e20424 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:02.959396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14024-a02f-4895-8379-5a5788e20424 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:02.959567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14024-a02f-4895-8379-5a5788e20424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:02.982860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d2b14024-a02f-4895-8379-5a5788e20424 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:02.985070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d2b14024-a02f-4895-8379-5a5788e20424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:03.015360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f18a4b-f5d3-4829-95c3-2f888d064db9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:03.059845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:03.060584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:03.060878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:03.061159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:03.061849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:03.062164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:03.063608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:03.069844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 490/999] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:18:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:03.073036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 510/1000] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:03.077971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:03.157702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:03.200129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb2d4a-dd8d-4981-aabe-5b100980f40e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:03.250988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:03.252013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:03.252510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:03.252998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:03.254172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:03.254714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:03.257382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:03.266977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 491/1001] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:18:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:03.273771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b14024-a02f-4895-8379-5a5788e20424 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:03.348489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:03.404748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:03.408926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b14024-a02f-4895-8379-5a5788e20424 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:03.449157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:03.449943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:03.450226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:03.451127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:03.451855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] RBAC: Adding query filter params (interface=public, service_id=d0ddf4592f044377aefd2fe7467ea3c2, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:03.452164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:03.453673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:03.459611 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 511/1002] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Oct 3 13:18:03 2022] GET /identity/v3/endpoints?service_id=d0ddf4592f044377aefd2fe7467ea3c2&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:03.466321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:03.471597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7a3d8-6358-45e0-8fed-ae2f83050ee3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:03.503714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:03.513777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:03.514428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:03.514708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:03.514981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:03.515577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:03.516955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:03.526503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:03.527619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:03.528565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 492/1003] 10.222.0.58 () {56 vars in 1126 bytes} [Mon Oct 3 13:18:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:03.533781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:03.618585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9a41f-cf56-4052-92a3-e97bb495a47d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:03.649871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5a805-c15f-49d8-b2e7-7d733d79388b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:03.663221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7619e80-680a-4bee-a860-5e360b2e5bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:03.664131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619e80-680a-4bee-a860-5e360b2e5bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:03.664537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619e80-680a-4bee-a860-5e360b2e5bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:03.664960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7619e80-680a-4bee-a860-5e360b2e5bd5 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:03.665889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7619e80-680a-4bee-a860-5e360b2e5bd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:03.667714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7619e80-680a-4bee-a860-5e360b2e5bd5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:03.676246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 512/1004] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:18:03 2022] GET /identity/v3/services => generated 1727 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:18:03.701792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:03.702619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:03.702901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:03.703180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:03.703852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:03.704156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:03.705666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:03.711834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 493/1005] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:18:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:03.717963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7619e80-680a-4bee-a860-5e360b2e5bd5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:03.752086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:03.835134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7619e80-680a-4bee-a860-5e360b2e5bd5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:03.873774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11195b48-1ad9-4392-b26a-a45cee277a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:03.874500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195b48-1ad9-4392-b26a-a45cee277a05 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:03.874886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195b48-1ad9-4392-b26a-a45cee277a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:03.875264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195b48-1ad9-4392-b26a-a45cee277a05 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:03.876015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11195b48-1ad9-4392-b26a-a45cee277a05 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:03.876423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11195b48-1ad9-4392-b26a-a45cee277a05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:03.878029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11195b48-1ad9-4392-b26a-a45cee277a05 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:03.891353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 513/1006] 10.222.0.58 () {56 vars in 1122 bytes} [Mon Oct 3 13:18:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:03.891553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:03.962040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:04.011937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1221de11-2699-4ec3-924d-2897c73e0419 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:04.052417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b63b8-2eae-4891-ad95-e96d108814d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:04.053509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b63b8-2eae-4891-ad95-e96d108814d5 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:04.053509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b63b8-2eae-4891-ad95-e96d108814d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:04.053926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b63b8-2eae-4891-ad95-e96d108814d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:04.054707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b63b8-2eae-4891-ad95-e96d108814d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:04.056173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b63b8-2eae-4891-ad95-e96d108814d5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:04.076457 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 494/1007] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:18:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:18:05.236529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219894d-603d-4446-a64c-d42079cb271f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:05.237062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219894d-603d-4446-a64c-d42079cb271f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:05.237143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219894d-603d-4446-a64c-d42079cb271f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:05.238613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 514/1008] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:05.243798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bd9a9-0eea-4654-8c54-bd0d580c772b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:05.244205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bd9a9-0eea-4654-8c54-bd0d580c772b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:05.244367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bd9a9-0eea-4654-8c54-bd0d580c772b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:05.263628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-548bd9a9-0eea-4654-8c54-bd0d580c772b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:05.264840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-548bd9a9-0eea-4654-8c54-bd0d580c772b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:05.349626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 495/1009] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:05.614761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219894d-603d-4446-a64c-d42079cb271f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:05.710314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:05.721433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219894d-603d-4446-a64c-d42079cb271f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:05.757072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde6a9f-6623-40cb-b715-04fc80d3d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:05.757733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6a9f-6623-40cb-b715-04fc80d3d964 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:05.758017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6a9f-6623-40cb-b715-04fc80d3d964 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:05.758301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde6a9f-6623-40cb-b715-04fc80d3d964 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:05.759001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde6a9f-6623-40cb-b715-04fc80d3d964 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:05.760450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde6a9f-6623-40cb-b715-04fc80d3d964 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:05.777035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 515/1010] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:18:05 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:18:05.964726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b69b-e01b-4687-bcb2-75dd823cdb7a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:05.965031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b69b-e01b-4687-bcb2-75dd823cdb7a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:05.965290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b69b-e01b-4687-bcb2-75dd823cdb7a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:05.968376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 496/1011] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:05.974266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcffd-2671-4ad9-a62a-a5ca88b5db6d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:05.974558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcffd-2671-4ad9-a62a-a5ca88b5db6d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:05.974839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dcffd-2671-4ad9-a62a-a5ca88b5db6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:05.994263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c42dcffd-2671-4ad9-a62a-a5ca88b5db6d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:05.995530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c42dcffd-2671-4ad9-a62a-a5ca88b5db6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:06.074918 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 516/1012] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:05 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:06.311211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece608-6693-4551-8a57-96681f93ad96 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:06.311800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece608-6693-4551-8a57-96681f93ad96 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:06.311800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ece608-6693-4551-8a57-96681f93ad96 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:06.313024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 497/1013] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:06.318679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9839-d959-43fd-97e2-13e5917243ac None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:06.319504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9839-d959-43fd-97e2-13e5917243ac None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:06.319984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de9839-d959-43fd-97e2-13e5917243ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:06.347391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ece608-6693-4551-8a57-96681f93ad96 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:06.348924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a1de9839-d959-43fd-97e2-13e5917243ac None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:06.350316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a1de9839-d959-43fd-97e2-13e5917243ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:06.402169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:06.430286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 517/1014] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:06 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:06.476492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ece608-6693-4551-8a57-96681f93ad96 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:06.518480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:06.519592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:06.520084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:06.520651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:06.521765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:06.523892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:06.537978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:06.539156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:06.540099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 498/1015] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:18:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:06.545213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de9839-d959-43fd-97e2-13e5917243ac None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:06.558521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:06.656369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de9839-d959-43fd-97e2-13e5917243ac None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:06.694493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:06.695104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:06.695411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:06.695655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:06.696281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:06.696585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:06.697985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:06.708759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 518/1016] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:18:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:06.709895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:06.718318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:06.803277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:06.822693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c124b2-c60b-4611-83d9-93d654e875f8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:06.831151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:06.838172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f3ae-ccff-44b3-bd24-d1033dee04f0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:06.862473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:06.863098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:06.863380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:06.863765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:06.864398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:06.864698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:06.866217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:06.877163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 499/1017] 10.222.0.58 () {56 vars in 1112 bytes} [Mon Oct 3 13:18:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:06.877364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c913272-c613-4b0a-bc5e-902a54d9e540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:06.877364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913272-c613-4b0a-bc5e-902a54d9e540 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:06.877364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913272-c613-4b0a-bc5e-902a54d9e540 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:06.877364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c913272-c613-4b0a-bc5e-902a54d9e540 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:06.878234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c913272-c613-4b0a-bc5e-902a54d9e540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:06.879036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:06.879685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c913272-c613-4b0a-bc5e-902a54d9e540 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:06.881695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 519/1018] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:06.887416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c913272-c613-4b0a-bc5e-902a54d9e540 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:06.996476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f582a71-9d6f-498a-8b15-e0555b94338e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:06.999680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c913272-c613-4b0a-bc5e-902a54d9e540 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:07.037285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e03517-2fd2-4667-9c7d-dccf031623ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:07.037698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e03517-2fd2-4667-9c7d-dccf031623ed None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:07.037962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e03517-2fd2-4667-9c7d-dccf031623ed None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:07.038238 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e03517-2fd2-4667-9c7d-dccf031623ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:07.038337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e04056-438e-4cb9-9392-ad1c7ed97b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:07.038914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e03517-2fd2-4667-9c7d-dccf031623ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:07.039014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e04056-438e-4cb9-9392-ad1c7ed97b4a None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:07.039244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e04056-438e-4cb9-9392-ad1c7ed97b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:07.039528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e04056-438e-4cb9-9392-ad1c7ed97b4a None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:07.040190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e04056-438e-4cb9-9392-ad1c7ed97b4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:07.040296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e03517-2fd2-4667-9c7d-dccf031623ed None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:07.041665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e04056-438e-4cb9-9392-ad1c7ed97b4a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:07.056028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:07.065564 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 500/1019] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:18:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:18:07.089531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 520/1020] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:18:06 2022] POST /identity/v3/projects => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:18:08.052908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f06c-d713-4945-8b3e-464978073b7a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:08.053621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f06c-d713-4945-8b3e-464978073b7a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:08.053621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4f06c-d713-4945-8b3e-464978073b7a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:08.055011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 501/1021] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:08.060652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f327c74-8d75-43d5-986b-b370c82d17a1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:08.061221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f327c74-8d75-43d5-986b-b370c82d17a1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:08.061221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f327c74-8d75-43d5-986b-b370c82d17a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:08.081780 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5f327c74-8d75-43d5-986b-b370c82d17a1 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:08.083026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5f327c74-8d75-43d5-986b-b370c82d17a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:08.167337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 521/1022] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:08.438441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4f06c-d713-4945-8b3e-464978073b7a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:08.472627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:08.555884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4f06c-d713-4945-8b3e-464978073b7a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:08.590039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:08.590593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:08.591447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:08.591731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] PATH_INFO: `/v3/services/image` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:08.592460 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:08.593859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:08.602521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:08.603513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:08.604381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 502/1023] 10.222.0.58 () {56 vars in 1105 bytes} [Mon Oct 3 13:18:08 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:08.617607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f327c74-8d75-43d5-986b-b370c82d17a1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:08.714885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f327c74-8d75-43d5-986b-b370c82d17a1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:08.732615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:08.752275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:08.752899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:08.753187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:08.753490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:08.754145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:08.754457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:08.756136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:08.766440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 522/1024] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:08.768620 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:08.866285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:08.878323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dddd1-3a55-4a13-85e7-918edbc52c00 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:08.916352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:08.916977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:08.917258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:08.917559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:08.918210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:08.918502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:08.920358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:08.929539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 503/1025] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:08.932362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:09.044883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b338a5-1236-4903-8af1-ad09243d1ff9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:09.080966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048a5d8-925e-4be3-a295-8eb43ef4b104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:09.081715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a5d8-925e-4be3-a295-8eb43ef4b104 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:09.081961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a5d8-925e-4be3-a295-8eb43ef4b104 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:09.082230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048a5d8-925e-4be3-a295-8eb43ef4b104 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:09.082987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048a5d8-925e-4be3-a295-8eb43ef4b104 None admin] RBAC: Adding query filter params (interface=public, service_id=5f6abbc4ba5b469e9e6b572eacc35a3d, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:09.083306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048a5d8-925e-4be3-a295-8eb43ef4b104 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:09.084681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048a5d8-925e-4be3-a295-8eb43ef4b104 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:09.095568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 523/1026] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Oct 3 13:18:08 2022] GET /identity/v3/endpoints?service_id=5f6abbc4ba5b469e9e6b572eacc35a3d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:09.096990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:09.203348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631bd95-4d95-4963-8473-384698fe17a0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:09.238651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72216652-0c7f-46de-aa33-207e0bf036a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:09.239284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72216652-0c7f-46de-aa33-207e0bf036a1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:09.239568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72216652-0c7f-46de-aa33-207e0bf036a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:09.239844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72216652-0c7f-46de-aa33-207e0bf036a1 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:09.240479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72216652-0c7f-46de-aa33-207e0bf036a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:09.241974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72216652-0c7f-46de-aa33-207e0bf036a1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:09.250201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 504/1027] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:18:09 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:18:09.396535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee52e71-20fd-419f-a7a9-1c2696fecccc None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:09.396918 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee52e71-20fd-419f-a7a9-1c2696fecccc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:09.397196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee52e71-20fd-419f-a7a9-1c2696fecccc None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:09.398718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 524/1028] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:09.405078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d77ed5-34a5-411c-b77d-28320d4b17e0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:09.405416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d77ed5-34a5-411c-b77d-28320d4b17e0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:09.405694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d77ed5-34a5-411c-b77d-28320d4b17e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:09.426357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-01d77ed5-34a5-411c-b77d-28320d4b17e0 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:09.427641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-01d77ed5-34a5-411c-b77d-28320d4b17e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:09.444037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:09.457143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883a7b-527d-493e-be00-bf5452fbae7e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:09.457345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883a7b-527d-493e-be00-bf5452fbae7e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:09.457651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883a7b-527d-493e-be00-bf5452fbae7e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:09.459147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 525/1029] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:18:09.464383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70db853-ee18-4473-9f99-d4299f0e3936 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:09.464947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70db853-ee18-4473-9f99-d4299f0e3936 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:09.465229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70db853-ee18-4473-9f99-d4299f0e3936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:09.487690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d70db853-ee18-4473-9f99-d4299f0e3936 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:09.488901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d70db853-ee18-4473-9f99-d4299f0e3936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:09.525474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 505/1030] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:09.539243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:09.567509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 526/1031] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:18:09.791177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d77ed5-34a5-411c-b77d-28320d4b17e0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:09.833394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70db853-ee18-4473-9f99-d4299f0e3936 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:09.912953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d77ed5-34a5-411c-b77d-28320d4b17e0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:09.951870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70db853-ee18-4473-9f99-d4299f0e3936 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:09.962683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1afc-5593-4582-8601-49e3ed319176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:09.963894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1afc-5593-4582-8601-49e3ed319176 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:09.964406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1afc-5593-4582-8601-49e3ed319176 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:09.964902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1afc-5593-4582-8601-49e3ed319176 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:09.966943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e1afc-5593-4582-8601-49e3ed319176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:09.969401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e1afc-5593-4582-8601-49e3ed319176 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:09.977886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 506/1032] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:09.979996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1afc-5593-4582-8601-49e3ed319176 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:09.996533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09066e-08e9-494c-86dc-433c3f0e38dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:09.998063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09066e-08e9-494c-86dc-433c3f0e38dc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:09.998358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09066e-08e9-494c-86dc-433c3f0e38dc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:09.998724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09066e-08e9-494c-86dc-433c3f0e38dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:10.002412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:10.012071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09066e-08e9-494c-86dc-433c3f0e38dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1b09066e-08e9-494c-86dc-433c3f0e38dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:10.017270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:10.018329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:10.019196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 527/1033] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Oct 3 13:18:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:10.024657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09066e-08e9-494c-86dc-433c3f0e38dc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:10.066599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:10.102565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1afc-5593-4582-8601-49e3ed319176 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:10.143541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09066e-08e9-494c-86dc-433c3f0e38dc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:10.161634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708d969-63eb-425b-83b1-e8ffb0a1b38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:10.162713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708d969-63eb-425b-83b1-e8ffb0a1b38c None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:10.163213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708d969-63eb-425b-83b1-e8ffb0a1b38c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:10.163713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708d969-63eb-425b-83b1-e8ffb0a1b38c None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:10.164778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708d969-63eb-425b-83b1-e8ffb0a1b38c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:10.167541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708d969-63eb-425b-83b1-e8ffb0a1b38c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:10.183823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:10.184945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:10.185239 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:10.185547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:10.186299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:10.186641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:10.188145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:10.199137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 528/1034] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Oct 3 13:18:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:10.206647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 507/1035] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:18:09 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:18:10.207980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:10.333955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:10.340410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21358142-5623-4374-8946-6a8fcecb5ebf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:10.388951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:10.389634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:10.389913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:10.390196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:10.405343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:10.406846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:18:10.424535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:10.426252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:18:10.427536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:18:10.427536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:10.427536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:10.427536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 529/1036] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:10.431571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708d969-63eb-425b-83b1-e8ffb0a1b38c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:10.516947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:10.555045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708d969-63eb-425b-83b1-e8ffb0a1b38c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:10.594215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:10.594753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:10.595033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:10.595306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:10.595957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:10.596267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:10.597846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:10.612062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 508/1037] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:10.617753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:10.732383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83275b21-a5d8-45db-8dae-2d220803d6b7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:10.770895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:10.771531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:10.771816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:10.772094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:10.780970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:10.782469 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:18:10.794242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:10.795366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:18:10.796360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:10.797795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 530/1038] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:18:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:10.802016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:10.915687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a017f-8c12-4fd4-89b6-271f5679b5de None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:10.948824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:10.955526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:10.956160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:10.956440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:10.956712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:10.957471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:10.957824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:10.959286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:10.984860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 509/1039] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:18:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:10.986851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:11.000806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:11.095973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef73f8-fc40-45f5-b96c-c9c056514867 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:11.130210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb911b54-505f-4fd4-ba1d-0f7e0f36694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:11.130834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb911b54-505f-4fd4-ba1d-0f7e0f36694d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:11.131113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb911b54-505f-4fd4-ba1d-0f7e0f36694d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:11.131387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb911b54-505f-4fd4-ba1d-0f7e0f36694d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:11.204567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb911b54-505f-4fd4-ba1d-0f7e0f36694d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:11.206069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb911b54-505f-4fd4-ba1d-0f7e0f36694d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:11.218199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:11.311919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 531/1040] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:18:11.317845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:11.432709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18624d90-de5d-4286-903f-70527c71f601 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:11.467535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0272b444-dc30-40e7-aa20-448c359474bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:11.468101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272b444-dc30-40e7-aa20-448c359474bb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:11.468384 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272b444-dc30-40e7-aa20-448c359474bb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:11.468660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0272b444-dc30-40e7-aa20-448c359474bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:11.479139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272b444-dc30-40e7-aa20-448c359474bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:11.480602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0272b444-dc30-40e7-aa20-448c359474bb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-0272b444-dc30-40e7-aa20-448c359474bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:18:11.500224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:11.501279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:18:11.507521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:18:11.507521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:11.507521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:11.507521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 510/1041] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:11.509117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb911b54-505f-4fd4-ba1d-0f7e0f36694d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:11.515625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c7f58-a820-47e3-8e51-85e812faf8ee None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:11.515907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c7f58-a820-47e3-8e51-85e812faf8ee None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:11.516187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c7f58-a820-47e3-8e51-85e812faf8ee None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:11.517891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 511/1042] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:18:11.522781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7c1d9-4e9a-4996-90a5-f127113f3d34 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:11.523074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7c1d9-4e9a-4996-90a5-f127113f3d34 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:11.523369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7c1d9-4e9a-4996-90a5-f127113f3d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:11.542785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c6e7c1d9-4e9a-4996-90a5-f127113f3d34 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:11.544038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c6e7c1d9-4e9a-4996-90a5-f127113f3d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:11.586891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:11.621798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb911b54-505f-4fd4-ba1d-0f7e0f36694d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:11.627218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 512/1043] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:18:11.638186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:11.663287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:11.663898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:11.664178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:11.664458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:11.665095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:11.665401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:11.666793 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:11.682985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 532/1044] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:11.684249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7c1d9-4e9a-4996-90a5-f127113f3d34 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:11.788687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7c1d9-4e9a-4996-90a5-f127113f3d34 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:11.823089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:11.823726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:11.824013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:11.824285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:11.831942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:11.833390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:18:11.845806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:11.846755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:11.847684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:18:11.847684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:18:11.847684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:18:11.847684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:18:11.847684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:18:11.847684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:18:11.847684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:18:11.847684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:11.849782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 513/1045] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:11.852712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:11.908048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:11.942704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:11.965351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c187e27-efe7-47b8-a325-bb287ba770de None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:12.003068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:12.003691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:12.003978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:12.004255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:12.005209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:12.005559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:12.007010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:12.019690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 533/1046] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:12.020005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cc471-2d5f-41c3-bea8-a61772d6fe66 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:12.025773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:12.057346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:12.062335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:12.063055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:12.063338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:12.063609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:12.064202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:12.065607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:12.076582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:12.077704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:12.078579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 514/1047] 10.222.0.58 () {56 vars in 1105 bytes} [Mon Oct 3 13:18:11 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:12.083802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:12.138310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6fd50-c287-4570-89ca-ffddc479962e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:12.176243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:12.176841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:12.177128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:12.177408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:12.178179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] RBAC: Adding query filter params (role.id=ebc793a410f84acbbe3e2ba7c6509a35, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c3ece3405a2b43288136ddb79b84a106) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:12.178492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:12.179950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:12.186570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 534/1048] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:18:12 2022] GET /identity/v3/role_assignments?user.id=c3ece3405a2b43288136ddb79b84a106&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=ebc793a410f84acbbe3e2ba7c6509a35 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:12.197705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:12.201108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb76405-f0a3-405a-a91e-9e28e5cd84b5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:12.241958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:12.242588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:12.242869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:12.243142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:12.243782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] RBAC: Adding query filter params (name=image) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:12.244111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:12.245583 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:12.252548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 515/1049] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:12.259020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:12.368433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:12.379768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f4f24-4aad-4d94-8ba0-16c61264311e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:12.418369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911d965-fc78-491e-a9a6-bcb46a5cdfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:12.419563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d965-fc78-491e-a9a6-bcb46a5cdfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:12.419848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d965-fc78-491e-a9a6-bcb46a5cdfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:12.420128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911d965-fc78-491e-a9a6-bcb46a5cdfe6 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:12.420774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911d965-fc78-491e-a9a6-bcb46a5cdfe6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:12.421085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911d965-fc78-491e-a9a6-bcb46a5cdfe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:12.422528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911d965-fc78-491e-a9a6-bcb46a5cdfe6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:12.432614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 535/1050] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:12.433558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:12.488881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd60f-accd-437f-830b-199ba5c01672 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:12.489162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd60f-accd-437f-830b-199ba5c01672 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:12.489473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd60f-accd-437f-830b-199ba5c01672 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:12.490972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 536/1051] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:18:12.496166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ec899-56c6-4c0b-a826-799fc0130e29 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:12.496455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ec899-56c6-4c0b-a826-799fc0130e29 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:12.496735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ec899-56c6-4c0b-a826-799fc0130e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:12.516593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-545ec899-56c6-4c0b-a826-799fc0130e29 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:12.517834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-545ec899-56c6-4c0b-a826-799fc0130e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:12.550714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55675c5-5f47-4c30-b6a6-462fafa8fc90 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:12.586751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0d29-6953-4bfa-9d1e-7ba59b13f298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:12.587502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 537/1052] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:18:12.587822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0d29-6953-4bfa-9d1e-7ba59b13f298 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:12.588113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0d29-6953-4bfa-9d1e-7ba59b13f298 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:12.588391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0d29-6953-4bfa-9d1e-7ba59b13f298 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:12.589066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce0d29-6953-4bfa-9d1e-7ba59b13f298 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:12.590509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce0d29-6953-4bfa-9d1e-7ba59b13f298 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:12.613072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 516/1053] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:18:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:18:12.862858 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ec899-56c6-4c0b-a826-799fc0130e29 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:12.958887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:12.978092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ec899-56c6-4c0b-a826-799fc0130e29 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:13.021781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40162033-2efd-467c-a21b-432a7bc1d191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:13.022420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40162033-2efd-467c-a21b-432a7bc1d191 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:13.022701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40162033-2efd-467c-a21b-432a7bc1d191 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:13.022976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40162033-2efd-467c-a21b-432a7bc1d191 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:13.033827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40162033-2efd-467c-a21b-432a7bc1d191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-40162033-2efd-467c-a21b-432a7bc1d191 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:13.038651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:13.040369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:13.042114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:13.047199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 538/1054] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:18:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:13.047772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0d29-6953-4bfa-9d1e-7ba59b13f298 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:13.068035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:13.164759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0d29-6953-4bfa-9d1e-7ba59b13f298 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:13.200412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:13.201253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:13.201563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:13.201843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:13.202512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:13.202818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:13.204191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:13.213158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 517/1055] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:18:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:13.219297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40162033-2efd-467c-a21b-432a7bc1d191 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:13.322772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40162033-2efd-467c-a21b-432a7bc1d191 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:13.366729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a873d-4361-4999-a2df-a99ef4d23095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:13.367366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a873d-4361-4999-a2df-a99ef4d23095 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:13.367650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a873d-4361-4999-a2df-a99ef4d23095 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:13.367926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a873d-4361-4999-a2df-a99ef4d23095 None admin] PATH_INFO: `/v3/users/f038a458166649e6bfe6d6608be4fd47` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:13.383284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a873d-4361-4999-a2df-a99ef4d23095 None admin] RBAC: Authorizing `identity:get_user(user_id=f038a458166649e6bfe6d6608be4fd47)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:13.384761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a873d-4361-4999-a2df-a99ef4d23095 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:13.391722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 539/1056] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:13 2022] GET /identity/v3/users/f038a458166649e6bfe6d6608be4fd47 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:13.392715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:13.479418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:13.511542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d93b62-7656-4466-92b1-2bf40e53e288 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:13.556376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd5fa-0ca0-4b3e-97df-454f8f7a9581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:13.556764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd5fa-0ca0-4b3e-97df-454f8f7a9581 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:13.557044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd5fa-0ca0-4b3e-97df-454f8f7a9581 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:13.557352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd5fa-0ca0-4b3e-97df-454f8f7a9581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:13.643459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd5fa-0ca0-4b3e-97df-454f8f7a9581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:13.644913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd5fa-0ca0-4b3e-97df-454f8f7a9581 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:13.725759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:13.761316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 518/1057] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 13:18:13.763526 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a873d-4361-4999-a2df-a99ef4d23095 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:13.766452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:13.887062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a873d-4361-4999-a2df-a99ef4d23095 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:13.927304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562f270-7bd1-440a-89ef-091d9dcfc211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:13.927948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f270-7bd1-440a-89ef-091d9dcfc211 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:13.928233 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f270-7bd1-440a-89ef-091d9dcfc211 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:13.928515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f270-7bd1-440a-89ef-091d9dcfc211 None admin] PATH_INFO: `/v3/projects/9f349d5a51154fd0aa903e4e226d21e0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:13.929693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562f270-7bd1-440a-89ef-091d9dcfc211 None admin] RBAC: Authorizing `identity:get_project(project_id=9f349d5a51154fd0aa903e4e226d21e0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:13.931154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562f270-7bd1-440a-89ef-091d9dcfc211 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:13.938430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 540/1058] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:13 2022] GET /identity/v3/projects/9f349d5a51154fd0aa903e4e226d21e0 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:13.940277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd5fa-0ca0-4b3e-97df-454f8f7a9581 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:14.050671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd5fa-0ca0-4b3e-97df-454f8f7a9581 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:14.088889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28d1f6-f378-4913-aa64-11613d53df05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:14.089549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28d1f6-f378-4913-aa64-11613d53df05 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:14.089840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28d1f6-f378-4913-aa64-11613d53df05 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:14.090116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28d1f6-f378-4913-aa64-11613d53df05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:14.091649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28d1f6-f378-4913-aa64-11613d53df05 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=9f349d5a51154fd0aa903e4e226d21e0, user.id=f038a458166649e6bfe6d6608be4fd47) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:14.092001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28d1f6-f378-4913-aa64-11613d53df05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:14.093569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28d1f6-f378-4913-aa64-11613d53df05 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:14.105492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 519/1059] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:18:13 2022] GET /identity/v3/role_assignments?user.id=f038a458166649e6bfe6d6608be4fd47&scope.project.id=9f349d5a51154fd0aa903e4e226d21e0&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:14.432587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4393218-818f-4511-81b1-2478e43f11d1 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:14.433149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4393218-818f-4511-81b1-2478e43f11d1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:14.433149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4393218-818f-4511-81b1-2478e43f11d1 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:14.435324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 541/1060] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:14.440964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75b6db-382d-4ca6-a86d-3b4331730bbc None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:14.441255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75b6db-382d-4ca6-a86d-3b4331730bbc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:14.441568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75b6db-382d-4ca6-a86d-3b4331730bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:14.461932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ee75b6db-382d-4ca6-a86d-3b4331730bbc None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:14.463130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ee75b6db-382d-4ca6-a86d-3b4331730bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:14.503937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:14.541501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 520/1061] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:14.823736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4393218-818f-4511-81b1-2478e43f11d1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:14.853582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b96ba7-2662-4ad5-80c8-0a12d6aa1f6a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:14.853876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b96ba7-2662-4ad5-80c8-0a12d6aa1f6a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:14.854148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b96ba7-2662-4ad5-80c8-0a12d6aa1f6a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:14.855495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 521/1062] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:18:14.862686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667c9c-0035-4705-9cb8-507a7b6e7e02 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:14.862992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667c9c-0035-4705-9cb8-507a7b6e7e02 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:14.863274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3667c9c-0035-4705-9cb8-507a7b6e7e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:14.884947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f3667c9c-0035-4705-9cb8-507a7b6e7e02 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:14.886201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f3667c9c-0035-4705-9cb8-507a7b6e7e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:14.902046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:14.938163 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4393218-818f-4511-81b1-2478e43f11d1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:14.963511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 522/1063] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:18:14.973945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc10ca-b67c-4625-9f2e-88c79f747739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:14.974591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc10ca-b67c-4625-9f2e-88c79f747739 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:14.974875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc10ca-b67c-4625-9f2e-88c79f747739 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:14.975152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc10ca-b67c-4625-9f2e-88c79f747739 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:14.988168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc10ca-b67c-4625-9f2e-88c79f747739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-63dc10ca-b67c-4625-9f2e-88c79f747739 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:14.993158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:14.994211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:14.998511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 542/1064] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Oct 3 13:18:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:15.001694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667c9c-0035-4705-9cb8-507a7b6e7e02 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.107758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3667c9c-0035-4705-9cb8-507a7b6e7e02 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:15.109286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:15.144868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:15.145507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:15.145795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:15.146072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:15.146740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:15.147051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:15.148425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:15.157248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 523/1065] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:15.163151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc10ca-b67c-4625-9f2e-88c79f747739 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.228227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.280768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc10ca-b67c-4625-9f2e-88c79f747739 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:15.314767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:15.323438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:15.324428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:15.324675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:15.324948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:15.335194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab90aa87-0210-405b-a626-58171e0dcbf7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:15.343197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:15.346631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:18:15.363153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:15.364179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:18:15.365102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:18:15.365102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:15.365102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:15.365102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 543/1066] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:15.370170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.383634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219938b-d068-4b02-9f41-96f490ee2a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:15.384123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219938b-d068-4b02-9f41-96f490ee2a0d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:15.384410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219938b-d068-4b02-9f41-96f490ee2a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:15.384689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219938b-d068-4b02-9f41-96f490ee2a0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:15.419983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:15.465263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219938b-d068-4b02-9f41-96f490ee2a0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:15.467605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219938b-d068-4b02-9f41-96f490ee2a0d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:15.494833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef4113-690c-434d-9eef-001761d2f23c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:15.534475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:15.535089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:15.535478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:15.535761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:15.536408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:15.536711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:15.538120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:15.551571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 544/1067] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:15.557105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.586961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 524/1068] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 13:18:15.587847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219938b-d068-4b02-9f41-96f490ee2a0d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.603490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:15.655071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:15.662926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4747f-0844-4b04-8bbc-771a49c92748 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:15.698102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:15.698746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:15.699038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:15.699321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:15.699565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219938b-d068-4b02-9f41-96f490ee2a0d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:15.708226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:15.709727 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:18:15.722521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:15.723687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:18:15.724842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:15.728720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 545/1069] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:15.729617 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.739872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:15.740934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:15.741390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:15.741984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:15.753782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:15.755999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:18:15.772494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:15.773601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:18:15.774540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:18:15.774540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:15.774540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:15.774540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 525/1070] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:15.780262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.836551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02936e25-c198-4068-ac36-594a3335f99a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:15.838196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:15.873040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:15.873693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:15.873979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:15.874259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:15.875026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:15.875328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:15.876770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:15.896829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 546/1071] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:15.897114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fbd9cc-1e87-40a0-9137-f718d4f1561d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:15.902680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:15.918500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:15.941028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:15.941662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:15.941952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:15.942253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:15.942871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:15.943177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:15.944558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:15.962884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 526/1072] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:16.018807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6ddf1-421a-42a3-acca-23b53c47598a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:16.061708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:16.062489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:16.062863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:16.063149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:16.074556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:16.076505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:18:16.095156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:16.096226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:18:16.101042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:18:16.101042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:16.101042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:16.101042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 547/1073] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:16.104248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:16.227060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437b32-29b8-449f-946d-6bd80eceaee8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:16.266583 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb08db-78f2-4978-8f1b-a9ec82f4caeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:16.267223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb08db-78f2-4978-8f1b-a9ec82f4caeb None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:16.267509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb08db-78f2-4978-8f1b-a9ec82f4caeb None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:16.267780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb08db-78f2-4978-8f1b-a9ec82f4caeb None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:16.268425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb08db-78f2-4978-8f1b-a9ec82f4caeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:16.268732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb08db-78f2-4978-8f1b-a9ec82f4caeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:16.270293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb08db-78f2-4978-8f1b-a9ec82f4caeb None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:16.283761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 527/1074] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:16.289886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:16.307778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:16.326212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d5d58-d7f8-4e79-a3be-5aa9cf1e340a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:16.326517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d5d58-d7f8-4e79-a3be-5aa9cf1e340a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:16.326787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d5d58-d7f8-4e79-a3be-5aa9cf1e340a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:16.328146 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 528/1075] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:16.335487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fbc88-2e3e-456d-a33d-6f0effb4f4a2 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:16.336442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fbc88-2e3e-456d-a33d-6f0effb4f4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:16.336728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fbc88-2e3e-456d-a33d-6f0effb4f4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:16.357691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-314fbc88-2e3e-456d-a33d-6f0effb4f4a2 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:16.358914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-314fbc88-2e3e-456d-a33d-6f0effb4f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:16.390418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:16.404016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2cdeb-1f7d-4d23-8148-ad6cdb7e938c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:16.430229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 529/1076] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:16.438736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:16.439371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:16.439656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:16.439965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:16.446638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:16.448085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:18:16.459879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:16.460765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:16.461879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 548/1077] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:16.467536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fbc88-2e3e-456d-a33d-6f0effb4f4a2 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:16.578854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fbc88-2e3e-456d-a33d-6f0effb4f4a2 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:16.613195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:16.613845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:16.614128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:16.614400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:16.615090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:16.615395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:16.616825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:16.629020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 530/1078] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:18:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:16.635277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:16.705260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:16.705884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:16.734240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:16.737948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84b9fe-7e20-465d-9b67-8ea0ddbf0214 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:16.781004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fda161-eeb2-447b-aa61-2a2ad43971d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:16.781709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fda161-eeb2-447b-aa61-2a2ad43971d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:16.781987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fda161-eeb2-447b-aa61-2a2ad43971d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:16.782264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fda161-eeb2-447b-aa61-2a2ad43971d0 None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c3ece3405a2b43288136ddb79b84a106/roles/ebc793a410f84acbbe3e2ba7c6509a35` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:16.784401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fda161-eeb2-447b-aa61-2a2ad43971d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=c3ece3405a2b43288136ddb79b84a106, role_id=ebc793a410f84acbbe3e2ba7c6509a35)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:16.786001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fda161-eeb2-447b-aa61-2a2ad43971d0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:16.799785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 549/1079] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:18:16 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/c3ece3405a2b43288136ddb79b84a106/roles/ebc793a410f84acbbe3e2ba7c6509a35 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:18:16.820597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d4c19-7ab2-4c9c-b332-41245eed4295 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:16.860080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea4dba-94b3-4f9f-8330-52c72a6084c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:16.860658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea4dba-94b3-4f9f-8330-52c72a6084c0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:16.860941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea4dba-94b3-4f9f-8330-52c72a6084c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:16.861218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea4dba-94b3-4f9f-8330-52c72a6084c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:16.877002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea4dba-94b3-4f9f-8330-52c72a6084c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-7bea4dba-94b3-4f9f-8330-52c72a6084c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:16.883613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:16.884823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:16.886058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 531/1080] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:16.891971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fda161-eeb2-447b-aa61-2a2ad43971d0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:17.004705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fda161-eeb2-447b-aa61-2a2ad43971d0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:17.051439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:17.057657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:17.058456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:17.058911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:17.059965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:17.060654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:17.060967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:17.062429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:17.071548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 550/1081] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:18:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:17.077485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea4dba-94b3-4f9f-8330-52c72a6084c0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:17.167346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:17.187916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea4dba-94b3-4f9f-8330-52c72a6084c0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:17.222657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2562-7eca-4a94-95ea-5eab11cbf68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:17.223302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2562-7eca-4a94-95ea-5eab11cbf68a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:17.223577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2562-7eca-4a94-95ea-5eab11cbf68a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:17.223849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd2562-7eca-4a94-95ea-5eab11cbf68a None admin] PATH_INFO: `/v3/users/f038a458166649e6bfe6d6608be4fd47` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:17.224973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd2562-7eca-4a94-95ea-5eab11cbf68a None admin] RBAC: Authorizing `identity:get_user(user_id=f038a458166649e6bfe6d6608be4fd47)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:17.226431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd2562-7eca-4a94-95ea-5eab11cbf68a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:17.228247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 532/1082] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:17 2022] GET /identity/v3/users/f038a458166649e6bfe6d6608be4fd47 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:17.234708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:17.337806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249eef1-b94a-4989-b490-606c84620049 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:17.376418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5195c03-8b90-4ef0-a803-9d36ac0bbfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:17.377062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5195c03-8b90-4ef0-a803-9d36ac0bbfee None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:17.377348 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5195c03-8b90-4ef0-a803-9d36ac0bbfee None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:17.377660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5195c03-8b90-4ef0-a803-9d36ac0bbfee None admin] PATH_INFO: `/v3/projects/9f349d5a51154fd0aa903e4e226d21e0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:17.378799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5195c03-8b90-4ef0-a803-9d36ac0bbfee None admin] RBAC: Authorizing `identity:get_project(project_id=9f349d5a51154fd0aa903e4e226d21e0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:17.380504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5195c03-8b90-4ef0-a803-9d36ac0bbfee None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:17.382228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 551/1083] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:17 2022] GET /identity/v3/projects/9f349d5a51154fd0aa903e4e226d21e0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:17.389030 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2562-7eca-4a94-95ea-5eab11cbf68a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:17.495544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd2562-7eca-4a94-95ea-5eab11cbf68a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:17.535597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e803d28-5aab-4ef6-8d46-ef0a00323a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:17.536278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e803d28-5aab-4ef6-8d46-ef0a00323a6b None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:17.536559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e803d28-5aab-4ef6-8d46-ef0a00323a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:17.536836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e803d28-5aab-4ef6-8d46-ef0a00323a6b None admin] PATH_INFO: `/v3/projects/9f349d5a51154fd0aa903e4e226d21e0/users/f038a458166649e6bfe6d6608be4fd47/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:17.538806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e803d28-5aab-4ef6-8d46-ef0a00323a6b None admin] RBAC: Authorizing `identity:create_grant(project_id=9f349d5a51154fd0aa903e4e226d21e0, user_id=f038a458166649e6bfe6d6608be4fd47, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:17.540246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e803d28-5aab-4ef6-8d46-ef0a00323a6b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:17.554872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 533/1084] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:18:17 2022] PUT /identity/v3/projects/9f349d5a51154fd0aa903e4e226d21e0/users/f038a458166649e6bfe6d6608be4fd47/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:18:19.018784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c1ec-5bdb-4546-a217-0bcbb0045828 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:19.019349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c1ec-5bdb-4546-a217-0bcbb0045828 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:19.019349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c1ec-5bdb-4546-a217-0bcbb0045828 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:19.020676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 552/1085] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:19.026064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5750e-2228-4961-a0c8-8d591ded8fb0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:19.026467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5750e-2228-4961-a0c8-8d591ded8fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:19.026774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5750e-2228-4961-a0c8-8d591ded8fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:19.045819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-24e5750e-2228-4961-a0c8-8d591ded8fb0 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:19.047010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-24e5750e-2228-4961-a0c8-8d591ded8fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:19.056158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:19.114759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 534/1086] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:18:19.388784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3c1ec-5bdb-4546-a217-0bcbb0045828 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:19.452930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:19.493600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3c1ec-5bdb-4546-a217-0bcbb0045828 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:19.529349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8bf3e-5d0c-4c63-9805-856781802716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:19.530023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8bf3e-5d0c-4c63-9805-856781802716 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:19.530310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8bf3e-5d0c-4c63-9805-856781802716 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:19.530594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8bf3e-5d0c-4c63-9805-856781802716 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:19.543046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8bf3e-5d0c-4c63-9805-856781802716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9bc8bf3e-5d0c-4c63-9805-856781802716 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:19.548496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:19.549652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:19.550531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 553/1087] 10.222.0.58 () {56 vars in 1120 bytes} [Mon Oct 3 13:18:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:19.555622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5750e-2228-4961-a0c8-8d591ded8fb0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:19.659384 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5750e-2228-4961-a0c8-8d591ded8fb0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:19.693073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe0e0f-c6f5-41f2-9794-165bea0c8c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:19.693714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0e0f-c6f5-41f2-9794-165bea0c8c13 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:19.693997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0e0f-c6f5-41f2-9794-165bea0c8c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:19.694273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe0e0f-c6f5-41f2-9794-165bea0c8c13 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:19.694938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe0e0f-c6f5-41f2-9794-165bea0c8c13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:19.695243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe0e0f-c6f5-41f2-9794-165bea0c8c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:19.696605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe0e0f-c6f5-41f2-9794-165bea0c8c13 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:19.705628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 535/1088] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Oct 3 13:18:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:19.711184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8bf3e-5d0c-4c63-9805-856781802716 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:19.743522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067c9e9-3401-4f93-af90-ea02109003bb None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:19.743807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067c9e9-3401-4f93-af90-ea02109003bb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:19.744080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067c9e9-3401-4f93-af90-ea02109003bb None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:19.745422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 536/1089] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:18:19.751631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fc3be-2775-467d-a386-c547c49bab6e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:19.751911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fc3be-2775-467d-a386-c547c49bab6e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:19.752187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fc3be-2775-467d-a386-c547c49bab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:19.765796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:19.772034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-eb9fc3be-2775-467d-a386-c547c49bab6e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:19.773407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-eb9fc3be-2775-467d-a386-c547c49bab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:19.823913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8bf3e-5d0c-4c63-9805-856781802716 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:19.854815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:19.855259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 537/1090] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:18:19.874832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:19.875483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:19.875770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:19.876049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:19.888844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:19.890364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:18:19.908117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:19.910186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:18:19.911404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:18:19.911404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:19.911404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:19.911404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 554/1091] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:19.918003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fc3be-2775-467d-a386-c547c49bab6e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.038765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fc3be-2775-467d-a386-c547c49bab6e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.075660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.076347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.076627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.076899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.077559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:20.077860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:20.079216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:20.081497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:20.091526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 538/1092] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:20.097124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.134033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.200185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e6fe2-e62d-43aa-a701-6fb649c4989e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.235251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.235882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.236413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.236413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.238994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0e99b-ae19-4883-899a-b6ed87f0a624 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.244420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:20.245909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:18:20.258646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:20.259716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:20.260693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 555/1093] 10.222.0.58 () {56 vars in 1093 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:20.266136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.277687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d70571-4529-418b-8db8-8747961a3107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.278356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d70571-4529-418b-8db8-8747961a3107 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.278639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d70571-4529-418b-8db8-8747961a3107 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.278910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d70571-4529-418b-8db8-8747961a3107 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.289893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d70571-4529-418b-8db8-8747961a3107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b1d70571-4529-418b-8db8-8747961a3107 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:20.294674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:20.295711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:20.296568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 539/1094] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:20.301521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:20.302003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d70571-4529-418b-8db8-8747961a3107 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.366316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:20.383456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d98616-99e1-45ba-8f8d-24c414fa763e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.413362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d70571-4529-418b-8db8-8747961a3107 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.418662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.419284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.419569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.419846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.420594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:20.420895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:20.422392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:20.440940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 556/1095] 10.222.0.58 () {56 vars in 1133 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:20.447008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.454637 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.455184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.455466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.455742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.456413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:20.456724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:20.458153 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:20.468350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 540/1096] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:20.474544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.549967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:20.559982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a10fa-1cf5-4d22-9ce2-05b2fcc797f4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.581435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e703793-1bc3-4bf0-b236-587ce03e9f00 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.596048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901828a-2dfc-466c-b870-4af8c70632ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.596663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901828a-2dfc-466c-b870-4af8c70632ea None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.596951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901828a-2dfc-466c-b870-4af8c70632ea None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.597230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901828a-2dfc-466c-b870-4af8c70632ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.600221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:20.618909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b0b2b-45d8-4a3e-a4b0-8a7ae9b95761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.619546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b0b2b-45d8-4a3e-a4b0-8a7ae9b95761 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.619829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b0b2b-45d8-4a3e-a4b0-8a7ae9b95761 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.620164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b0b2b-45d8-4a3e-a4b0-8a7ae9b95761 None admin] PATH_INFO: `/v3/users/f038a458166649e6bfe6d6608be4fd47` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.621265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b0b2b-45d8-4a3e-a4b0-8a7ae9b95761 None admin] RBAC: Authorizing `identity:get_user(user_id=f038a458166649e6bfe6d6608be4fd47)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:20.623215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b0b2b-45d8-4a3e-a4b0-8a7ae9b95761 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:20.624842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 541/1097] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/users/f038a458166649e6bfe6d6608be4fd47 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:20.632143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b0b2b-45d8-4a3e-a4b0-8a7ae9b95761 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.672730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901828a-2dfc-466c-b870-4af8c70632ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:20.674265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901828a-2dfc-466c-b870-4af8c70632ea None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:20.740972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b0b2b-45d8-4a3e-a4b0-8a7ae9b95761 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.770718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:20.780302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409d5ab-ec7a-44ce-b310-72671e032785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.780944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409d5ab-ec7a-44ce-b310-72671e032785 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.781231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409d5ab-ec7a-44ce-b310-72671e032785 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.781543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409d5ab-ec7a-44ce-b310-72671e032785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.783092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 557/1098] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 13:18:20.789205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901828a-2dfc-466c-b870-4af8c70632ea None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.844289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:20.861601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409d5ab-ec7a-44ce-b310-72671e032785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:20.863081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409d5ab-ec7a-44ce-b310-72671e032785 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:20.899705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901828a-2dfc-466c-b870-4af8c70632ea None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:20.935718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:20.936360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:20.936705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:20.936989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:20.947394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:20.949259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:18:20.964592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:20.966263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:18:20.967525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:18:20.967525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:18:20.967525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:20.967525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 558/1099] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:18:20.967823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 542/1100] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 13:18:20.971587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:20.972750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409d5ab-ec7a-44ce-b310-72671e032785 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:21.019276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:21.058450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:21.073387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409d5ab-ec7a-44ce-b310-72671e032785 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:21.077494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d306219-9fd2-4b2d-97ee-5303f8fd4208 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:21.108976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea90b6-9e79-427d-a12f-f782930f6437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:21.109652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea90b6-9e79-427d-a12f-f782930f6437 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:21.109935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea90b6-9e79-427d-a12f-f782930f6437 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:21.110207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea90b6-9e79-427d-a12f-f782930f6437 None admin] PATH_INFO: `/v3/projects/9f349d5a51154fd0aa903e4e226d21e0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:21.111325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea90b6-9e79-427d-a12f-f782930f6437 None admin] RBAC: Authorizing `identity:get_project(project_id=9f349d5a51154fd0aa903e4e226d21e0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:21.112758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea90b6-9e79-427d-a12f-f782930f6437 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:21.113335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:21.113987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:21.114273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:21.114706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:21.114816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 543/1101] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/projects/9f349d5a51154fd0aa903e4e226d21e0 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:21.115291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:21.115675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:21.117059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:21.120466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea90b6-9e79-427d-a12f-f782930f6437 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:21.135553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 559/1102] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:21.136646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:21.221481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea90b6-9e79-427d-a12f-f782930f6437 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:21.240174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acd731-99a3-4235-b41e-6289e8cb2b8a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:21.242939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:21.257128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:21.257770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:21.258060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:21.258336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:21.259063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=9f349d5a51154fd0aa903e4e226d21e0, user.id=f038a458166649e6bfe6d6608be4fd47) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:21.259377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:21.260820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:21.266716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 544/1103] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:18:21 2022] GET /identity/v3/role_assignments?user.id=f038a458166649e6bfe6d6608be4fd47&scope.project.id=9f349d5a51154fd0aa903e4e226d21e0&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:18:21.275079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:21.275655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:21.275940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:21.276218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:21.284844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:21.286396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:18:21.299683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:21.300740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:21.301702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 560/1104] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:18:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:21.307062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:21.328587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:21.426489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9fe8-a063-4b30-bdad-89e05c51e24f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:21.471471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b47e5-7519-40ec-8861-29199238dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:21.472084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b47e5-7519-40ec-8861-29199238dd9b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:21.472361 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b47e5-7519-40ec-8861-29199238dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:21.472631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b47e5-7519-40ec-8861-29199238dd9b None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:21.473325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b47e5-7519-40ec-8861-29199238dd9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:21.473656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b47e5-7519-40ec-8861-29199238dd9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:21.475085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b47e5-7519-40ec-8861-29199238dd9b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:21.487640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 545/1105] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:21.493946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:21.598917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960eaae-da49-4777-8aed-f5a859267e69 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:21.637098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa5b2a-f415-4154-804c-833eb8836cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:21.637678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5b2a-f415-4154-804c-833eb8836cbe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:21.638077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5b2a-f415-4154-804c-833eb8836cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:21.638360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5b2a-f415-4154-804c-833eb8836cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:21.639088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa5b2a-f415-4154-804c-833eb8836cbe None admin] RBAC: Adding query filter params (role.id=ebc793a410f84acbbe3e2ba7c6509a35, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=c3ece3405a2b43288136ddb79b84a106) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:21.639404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa5b2a-f415-4154-804c-833eb8836cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:21.640836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa5b2a-f415-4154-804c-833eb8836cbe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:21.646708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 561/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:18:21 2022] GET /identity/v3/role_assignments?user.id=c3ece3405a2b43288136ddb79b84a106&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=ebc793a410f84acbbe3e2ba7c6509a35 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:23.467047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c18ed3-2741-49e0-a69c-d82cdc38a1c0 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:23.467596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c18ed3-2741-49e0-a69c-d82cdc38a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:23.467596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c18ed3-2741-49e0-a69c-d82cdc38a1c0 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:23.468923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 546/1107] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:23.474358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5536d33-40b3-481d-a50c-612ae65292ef None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:23.474807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5536d33-40b3-481d-a50c-612ae65292ef None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:23.474926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5536d33-40b3-481d-a50c-612ae65292ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:23.494303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c5536d33-40b3-481d-a50c-612ae65292ef None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:23.495539 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c5536d33-40b3-481d-a50c-612ae65292ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:23.518583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:23.568812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 562/1108] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:23.841126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c18ed3-2741-49e0-a69c-d82cdc38a1c0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:23.920258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:23.942407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c18ed3-2741-49e0-a69c-d82cdc38a1c0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:23.977350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683f9dc-8ac8-4f40-8fe7-3363105c8775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:23.978012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683f9dc-8ac8-4f40-8fe7-3363105c8775 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:23.978295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683f9dc-8ac8-4f40-8fe7-3363105c8775 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:23.978571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683f9dc-8ac8-4f40-8fe7-3363105c8775 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:23.979690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683f9dc-8ac8-4f40-8fe7-3363105c8775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:23.981369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683f9dc-8ac8-4f40-8fe7-3363105c8775 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:23.983050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 547/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:23.988843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5536d33-40b3-481d-a50c-612ae65292ef None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:24.087040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5536d33-40b3-481d-a50c-612ae65292ef None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:24.120443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfe1d5-eadd-47aa-b016-aa8d0b846ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:24.120975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfe1d5-eadd-47aa-b016-aa8d0b846ba6 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:24.121259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfe1d5-eadd-47aa-b016-aa8d0b846ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:24.122184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfe1d5-eadd-47aa-b016-aa8d0b846ba6 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:24.122958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfe1d5-eadd-47aa-b016-aa8d0b846ba6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:24.124401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfe1d5-eadd-47aa-b016-aa8d0b846ba6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:24.145300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 563/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:18:23 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:18:26.390607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ea0b-9584-483c-a2f5-dd7e40e97526 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:26.391137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ea0b-9584-483c-a2f5-dd7e40e97526 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:26.391137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ea0b-9584-483c-a2f5-dd7e40e97526 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:26.392512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 548/1111] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:18:26.397990 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0cbf-320d-4c83-841d-dc806ef16248 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:26.398410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0cbf-320d-4c83-841d-dc806ef16248 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:26.398551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e0cbf-320d-4c83-841d-dc806ef16248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:26.417289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-024e0cbf-320d-4c83-841d-dc806ef16248 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:26.419478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-024e0cbf-320d-4c83-841d-dc806ef16248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:26.429071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:26.489549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 564/1112] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:26.758989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88ea0b-9584-483c-a2f5-dd7e40e97526 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:26.869558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88ea0b-9584-483c-a2f5-dd7e40e97526 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:26.906507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde19a3-b13e-4837-9402-253c498f1e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:26.907147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde19a3-b13e-4837-9402-253c498f1e4a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:26.907428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde19a3-b13e-4837-9402-253c498f1e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:26.907702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde19a3-b13e-4837-9402-253c498f1e4a None admin] PATH_INFO: `/v3/roles/753d3bbfcdaa4cef86cc8de37ae19906` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:26.909114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde19a3-b13e-4837-9402-253c498f1e4a None admin] RBAC: Authorizing `identity:get_role(role_id=753d3bbfcdaa4cef86cc8de37ae19906)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:26.910525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde19a3-b13e-4837-9402-253c498f1e4a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:26.911874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 549/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:26 2022] GET /identity/v3/roles/753d3bbfcdaa4cef86cc8de37ae19906 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:26.917797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e0cbf-320d-4c83-841d-dc806ef16248 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:27.018777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e0cbf-320d-4c83-841d-dc806ef16248 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:27.052202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4038-c043-4248-990a-4935b25894a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:27.052835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4038-c043-4248-990a-4935b25894a9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:27.053123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4038-c043-4248-990a-4935b25894a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:27.053403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce4038-c043-4248-990a-4935b25894a9 None admin] PATH_INFO: `/v3/users/71b30f26155a486c8b4f0387f39892de` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:27.067409 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce4038-c043-4248-990a-4935b25894a9 None admin] RBAC: Authorizing `identity:get_user(user_id=71b30f26155a486c8b4f0387f39892de)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:27.068880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce4038-c043-4248-990a-4935b25894a9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:27.070556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 565/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:26 2022] GET /identity/v3/users/71b30f26155a486c8b4f0387f39892de => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:27.077128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde19a3-b13e-4837-9402-253c498f1e4a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:27.093227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:27.179123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde19a3-b13e-4837-9402-253c498f1e4a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:27.214074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174b329-df24-499a-a692-9babb8b2d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:27.214692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174b329-df24-499a-a692-9babb8b2d16f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:27.214998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174b329-df24-499a-a692-9babb8b2d16f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:27.215333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174b329-df24-499a-a692-9babb8b2d16f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:27.287230 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174b329-df24-499a-a692-9babb8b2d16f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:27.288631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174b329-df24-499a-a692-9babb8b2d16f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:27.304900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:27.396532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 550/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:18:27.402706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4038-c043-4248-990a-4935b25894a9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:27.406476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:27.504502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce4038-c043-4248-990a-4935b25894a9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:27.538667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6eaf91-3621-4610-81e5-5c9fcc0a7bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:27.539221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eaf91-3621-4610-81e5-5c9fcc0a7bfe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:27.539506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eaf91-3621-4610-81e5-5c9fcc0a7bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:27.539785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6eaf91-3621-4610-81e5-5c9fcc0a7bfe None admin] PATH_INFO: `/v3/projects/9f349d5a51154fd0aa903e4e226d21e0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:27.540906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eaf91-3621-4610-81e5-5c9fcc0a7bfe None admin] RBAC: Authorizing `identity:get_project(project_id=9f349d5a51154fd0aa903e4e226d21e0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:27.542383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6eaf91-3621-4610-81e5-5c9fcc0a7bfe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:27.544582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 566/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:27 2022] GET /identity/v3/projects/9f349d5a51154fd0aa903e4e226d21e0 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:27.550987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174b329-df24-499a-a692-9babb8b2d16f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:27.654261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174b329-df24-499a-a692-9babb8b2d16f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:27.668616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:27.689487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa6ef0-a42f-4263-a9ac-771b24092c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:27.690089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa6ef0-a42f-4263-a9ac-771b24092c85 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:27.690372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa6ef0-a42f-4263-a9ac-771b24092c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:27.690648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa6ef0-a42f-4263-a9ac-771b24092c85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:27.691376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa6ef0-a42f-4263-a9ac-771b24092c85 None admin] RBAC: Adding query filter params (role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=9f349d5a51154fd0aa903e4e226d21e0, user.id=71b30f26155a486c8b4f0387f39892de) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:27.691685 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa6ef0-a42f-4263-a9ac-771b24092c85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:27.693112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa6ef0-a42f-4263-a9ac-771b24092c85 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:27.698472 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 551/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:18:27 2022] GET /identity/v3/role_assignments?user.id=71b30f26155a486c8b4f0387f39892de&scope.project.id=9f349d5a51154fd0aa903e4e226d21e0&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:29.855973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f55225-8751-4aaf-934e-8a3f8a9fe0c6 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:29.856512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f55225-8751-4aaf-934e-8a3f8a9fe0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:29.856635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f55225-8751-4aaf-934e-8a3f8a9fe0c6 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:29.858032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 567/1118] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:29.863334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10fab-bf27-446d-a318-54561fb4105a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:29.863776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10fab-bf27-446d-a318-54561fb4105a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:29.863906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10fab-bf27-446d-a318-54561fb4105a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:29.882213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-74e10fab-bf27-446d-a318-54561fb4105a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:29.883411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74e10fab-bf27-446d-a318-54561fb4105a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:29.954671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 552/1119] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:18:30.232245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f55225-8751-4aaf-934e-8a3f8a9fe0c6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:30.303112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:30.335216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f55225-8751-4aaf-934e-8a3f8a9fe0c6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:30.370972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c736b5-7104-4ce1-876e-4a958e00aef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:30.371733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c736b5-7104-4ce1-876e-4a958e00aef7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:30.372020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c736b5-7104-4ce1-876e-4a958e00aef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:30.372298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c736b5-7104-4ce1-876e-4a958e00aef7 None admin] PATH_INFO: `/v3/roles/753d3bbfcdaa4cef86cc8de37ae19906` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:30.373747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c736b5-7104-4ce1-876e-4a958e00aef7 None admin] RBAC: Authorizing `identity:get_role(role_id=753d3bbfcdaa4cef86cc8de37ae19906)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:30.375129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c736b5-7104-4ce1-876e-4a958e00aef7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:30.376478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 568/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:30 2022] GET /identity/v3/roles/753d3bbfcdaa4cef86cc8de37ae19906 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:30.382617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e10fab-bf27-446d-a318-54561fb4105a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:30.480661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e10fab-bf27-446d-a318-54561fb4105a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:30.490385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:30.516910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf1d3b-60ab-4cf0-a889-08f75dfd9b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:30.517574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf1d3b-60ab-4cf0-a889-08f75dfd9b0a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:30.517854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf1d3b-60ab-4cf0-a889-08f75dfd9b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:30.518122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf1d3b-60ab-4cf0-a889-08f75dfd9b0a None admin] PATH_INFO: `/v3/users/71b30f26155a486c8b4f0387f39892de` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:30.519238 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf1d3b-60ab-4cf0-a889-08f75dfd9b0a None admin] RBAC: Authorizing `identity:get_user(user_id=71b30f26155a486c8b4f0387f39892de)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:30.520670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf1d3b-60ab-4cf0-a889-08f75dfd9b0a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:30.522303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 553/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:30 2022] GET /identity/v3/users/71b30f26155a486c8b4f0387f39892de => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:30.529050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c736b5-7104-4ce1-876e-4a958e00aef7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:30.632050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c736b5-7104-4ce1-876e-4a958e00aef7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:30.668848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fba26-8fae-4156-8f89-27d2b09ea2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:30.669537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fba26-8fae-4156-8f89-27d2b09ea2bf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:30.669821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fba26-8fae-4156-8f89-27d2b09ea2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:30.670100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fba26-8fae-4156-8f89-27d2b09ea2bf None admin] PATH_INFO: `/v3/projects/9f349d5a51154fd0aa903e4e226d21e0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:30.671278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fba26-8fae-4156-8f89-27d2b09ea2bf None admin] RBAC: Authorizing `identity:get_project(project_id=9f349d5a51154fd0aa903e4e226d21e0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:30.672737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fba26-8fae-4156-8f89-27d2b09ea2bf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:30.674482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 569/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:30 2022] GET /identity/v3/projects/9f349d5a51154fd0aa903e4e226d21e0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:30.680753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf1d3b-60ab-4cf0-a889-08f75dfd9b0a None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:30.785913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf1d3b-60ab-4cf0-a889-08f75dfd9b0a None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:30.820984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a0734-6626-4286-b104-a8c28fb822ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:30.821674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0734-6626-4286-b104-a8c28fb822ce None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:30.821958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0734-6626-4286-b104-a8c28fb822ce None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:30.822235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0734-6626-4286-b104-a8c28fb822ce None admin] PATH_INFO: `/v3/projects/9f349d5a51154fd0aa903e4e226d21e0/users/71b30f26155a486c8b4f0387f39892de/roles/753d3bbfcdaa4cef86cc8de37ae19906` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:30.824518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a0734-6626-4286-b104-a8c28fb822ce None admin] RBAC: Authorizing `identity:create_grant(project_id=9f349d5a51154fd0aa903e4e226d21e0, user_id=71b30f26155a486c8b4f0387f39892de, role_id=753d3bbfcdaa4cef86cc8de37ae19906)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:30.826308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a0734-6626-4286-b104-a8c28fb822ce None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:30.838717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 554/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:18:30 2022] PUT /identity/v3/projects/9f349d5a51154fd0aa903e4e226d21e0/users/71b30f26155a486c8b4f0387f39892de/roles/753d3bbfcdaa4cef86cc8de37ae19906 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:18:33.078242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c68f0-2d33-42d8-983d-12288bcf1ced None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:33.078781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c68f0-2d33-42d8-983d-12288bcf1ced None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:33.078781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c68f0-2d33-42d8-983d-12288bcf1ced None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:33.080139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 570/1124] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:33.085332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd6a2-b2eb-4949-85f0-07f467b1ea9a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:33.085778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd6a2-b2eb-4949-85f0-07f467b1ea9a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:33.085920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd6a2-b2eb-4949-85f0-07f467b1ea9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:33.104641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-762bd6a2-b2eb-4949-85f0-07f467b1ea9a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:33.106797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-762bd6a2-b2eb-4949-85f0-07f467b1ea9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:33.135118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:33.175366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 555/1125] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:33.452307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c68f0-2d33-42d8-983d-12288bcf1ced None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:33.462005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:33.564969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c68f0-2d33-42d8-983d-12288bcf1ced None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:33.602047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af48d51-a2f4-4d79-b7c7-3a595811ac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:33.602595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af48d51-a2f4-4d79-b7c7-3a595811ac1e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:33.602877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af48d51-a2f4-4d79-b7c7-3a595811ac1e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:33.603156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af48d51-a2f4-4d79-b7c7-3a595811ac1e None admin] PATH_INFO: `/v3/roles/753d3bbfcdaa4cef86cc8de37ae19906` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:33.604572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af48d51-a2f4-4d79-b7c7-3a595811ac1e None admin] RBAC: Authorizing `identity:get_role(role_id=753d3bbfcdaa4cef86cc8de37ae19906)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:33.606043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af48d51-a2f4-4d79-b7c7-3a595811ac1e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:33.607555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 571/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:33 2022] GET /identity/v3/roles/753d3bbfcdaa4cef86cc8de37ae19906 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:33.613281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd6a2-b2eb-4949-85f0-07f467b1ea9a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:33.711699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd6a2-b2eb-4949-85f0-07f467b1ea9a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:33.745790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d764-b2ff-46d7-80ed-0ad77cef9ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:33.747222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d764-b2ff-46d7-80ed-0ad77cef9ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:33.747438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d764-b2ff-46d7-80ed-0ad77cef9ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:33.747811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38d764-b2ff-46d7-80ed-0ad77cef9ad3 None admin] PATH_INFO: `/v3/users/71b30f26155a486c8b4f0387f39892de` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:33.749007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38d764-b2ff-46d7-80ed-0ad77cef9ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=71b30f26155a486c8b4f0387f39892de)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:33.750468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38d764-b2ff-46d7-80ed-0ad77cef9ad3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:33.752053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 556/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:33 2022] GET /identity/v3/users/71b30f26155a486c8b4f0387f39892de => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:33.758226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af48d51-a2f4-4d79-b7c7-3a595811ac1e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:33.834047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:33.858939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af48d51-a2f4-4d79-b7c7-3a595811ac1e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:33.893711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d79b7-114b-4410-a744-d1703e17a5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:33.894330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d79b7-114b-4410-a744-d1703e17a5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:33.894612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d79b7-114b-4410-a744-d1703e17a5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:33.894889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d79b7-114b-4410-a744-d1703e17a5a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:33.966705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d79b7-114b-4410-a744-d1703e17a5a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:33.968183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d79b7-114b-4410-a744-d1703e17a5a5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:34.045473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:34.072610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 572/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:18:34.079540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d764-b2ff-46d7-80ed-0ad77cef9ad3 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:34.114009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:34.184519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38d764-b2ff-46d7-80ed-0ad77cef9ad3 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:34.218452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aef7c92-509a-4322-933a-28a8e7e350e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:34.219091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7c92-509a-4322-933a-28a8e7e350e7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:34.219375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7c92-509a-4322-933a-28a8e7e350e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:34.219652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7c92-509a-4322-933a-28a8e7e350e7 None admin] PATH_INFO: `/v3/projects/9f349d5a51154fd0aa903e4e226d21e0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:34.220876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef7c92-509a-4322-933a-28a8e7e350e7 None admin] RBAC: Authorizing `identity:get_project(project_id=9f349d5a51154fd0aa903e4e226d21e0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:34.222352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aef7c92-509a-4322-933a-28a8e7e350e7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:34.225413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 557/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:34 2022] GET /identity/v3/projects/9f349d5a51154fd0aa903e4e226d21e0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:34.233355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d79b7-114b-4410-a744-d1703e17a5a5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:34.343120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d79b7-114b-4410-a744-d1703e17a5a5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:34.379985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0183eb-fd00-4c32-85a3-6ce49748e45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:34.380523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0183eb-fd00-4c32-85a3-6ce49748e45f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:34.380804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0183eb-fd00-4c32-85a3-6ce49748e45f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:34.381079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0183eb-fd00-4c32-85a3-6ce49748e45f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:34.381853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0183eb-fd00-4c32-85a3-6ce49748e45f None admin] RBAC: Adding query filter params (role.id=753d3bbfcdaa4cef86cc8de37ae19906, scope.project.id=9f349d5a51154fd0aa903e4e226d21e0, user.id=71b30f26155a486c8b4f0387f39892de) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:34.382166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0183eb-fd00-4c32-85a3-6ce49748e45f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:34.383633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0183eb-fd00-4c32-85a3-6ce49748e45f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:34.390770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 573/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:18:34 2022] GET /identity/v3/role_assignments?user.id=71b30f26155a486c8b4f0387f39892de&scope.project.id=9f349d5a51154fd0aa903e4e226d21e0&role.id=753d3bbfcdaa4cef86cc8de37ae19906 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:36.641950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb287-89b6-4670-9e5b-639c57a4f6f1 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:36.642447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb287-89b6-4670-9e5b-639c57a4f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:36.642447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb287-89b6-4670-9e5b-639c57a4f6f1 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:36.643819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 558/1131] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:36.649163 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f3ba6-5b3e-4a41-82c4-40a779c13289 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:36.649643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f3ba6-5b3e-4a41-82c4-40a779c13289 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:36.649743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f3ba6-5b3e-4a41-82c4-40a779c13289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:36.668437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b05f3ba6-5b3e-4a41-82c4-40a779c13289 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:36.669659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b05f3ba6-5b3e-4a41-82c4-40a779c13289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:36.697925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:36.735044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 574/1132] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:37.007780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcb287-89b6-4670-9e5b-639c57a4f6f1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:37.109774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcb287-89b6-4670-9e5b-639c57a4f6f1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:37.111299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:37.145912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd28b7a-48ee-4ec9-bec1-31bd5b60ae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:37.146543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd28b7a-48ee-4ec9-bec1-31bd5b60ae3a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:37.146821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd28b7a-48ee-4ec9-bec1-31bd5b60ae3a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:37.147088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd28b7a-48ee-4ec9-bec1-31bd5b60ae3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:37.148191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd28b7a-48ee-4ec9-bec1-31bd5b60ae3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:37.149831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd28b7a-48ee-4ec9-bec1-31bd5b60ae3a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:37.151547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 559/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:37.158993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f3ba6-5b3e-4a41-82c4-40a779c13289 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:37.258778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f3ba6-5b3e-4a41-82c4-40a779c13289 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:37.291972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f9e87-c981-4bf4-b56a-12603c9ec13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:37.292587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9e87-c981-4bf4-b56a-12603c9ec13a None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:37.293386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9e87-c981-4bf4-b56a-12603c9ec13a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:37.293703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9e87-c981-4bf4-b56a-12603c9ec13a None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:37.294536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f9e87-c981-4bf4-b56a-12603c9ec13a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:37.295969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f9e87-c981-4bf4-b56a-12603c9ec13a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:37.323109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 575/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:18:37 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:18:39.541317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f8e16-e647-4747-a7f4-b2accb81b854 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:39.541916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f8e16-e647-4747-a7f4-b2accb81b854 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:39.541916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f8e16-e647-4747-a7f4-b2accb81b854 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:39.543846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 560/1135] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:39.549991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b3134-d9f2-41ce-870a-15448068afbe None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:39.550534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b3134-d9f2-41ce-870a-15448068afbe None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:39.550534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b3134-d9f2-41ce-870a-15448068afbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:39.570035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-372b3134-d9f2-41ce-870a-15448068afbe None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:39.571266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-372b3134-d9f2-41ce-870a-15448068afbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:39.587119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:39.641134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 576/1136] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:39.916027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f8e16-e647-4747-a7f4-b2accb81b854 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:40.021869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f8e16-e647-4747-a7f4-b2accb81b854 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:40.063335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4d5ae-8e3e-4d40-9508-e9b4f5b7a153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:40.064006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4d5ae-8e3e-4d40-9508-e9b4f5b7a153 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:40.064294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4d5ae-8e3e-4d40-9508-e9b4f5b7a153 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:40.064585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4d5ae-8e3e-4d40-9508-e9b4f5b7a153 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:40.065773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4d5ae-8e3e-4d40-9508-e9b4f5b7a153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:40.067233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4d5ae-8e3e-4d40-9508-e9b4f5b7a153 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:40.068843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 561/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:18:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:40.074994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b3134-d9f2-41ce-870a-15448068afbe None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:40.178975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b3134-d9f2-41ce-870a-15448068afbe None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:40.213253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01df86-30cc-4d63-9abf-23962b1af6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:40.213931 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01df86-30cc-4d63-9abf-23962b1af6c3 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:40.214221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01df86-30cc-4d63-9abf-23962b1af6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:40.215046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01df86-30cc-4d63-9abf-23962b1af6c3 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:40.215760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01df86-30cc-4d63-9abf-23962b1af6c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:40.217248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01df86-30cc-4d63-9abf-23962b1af6c3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:40.240681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 577/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:18:40 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:18:42.491789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e334a-e456-4fa6-950c-327222aedf09 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:42.492350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e334a-e456-4fa6-950c-327222aedf09 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:42.492350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e334a-e456-4fa6-950c-327222aedf09 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:42.493704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 562/1139] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:42.499325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f740ec-b213-49ed-aa30-10c7ed1a31cd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:42.499775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f740ec-b213-49ed-aa30-10c7ed1a31cd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:42.499875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f740ec-b213-49ed-aa30-10c7ed1a31cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:42.514152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:42.520047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-82f740ec-b213-49ed-aa30-10c7ed1a31cd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:42.521736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-82f740ec-b213-49ed-aa30-10c7ed1a31cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:42.590423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 578/1140] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:42.858660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e334a-e456-4fa6-950c-327222aedf09 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:42.904100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:42.966917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e334a-e456-4fa6-950c-327222aedf09 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:43.004893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02207b21-7381-474d-beaa-825640220fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:43.005570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207b21-7381-474d-beaa-825640220fef None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:43.005852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207b21-7381-474d-beaa-825640220fef None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:43.006124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207b21-7381-474d-beaa-825640220fef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:43.017526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02207b21-7381-474d-beaa-825640220fef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-02207b21-7381-474d-beaa-825640220fef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:43.022282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:43.023490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:43.024463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 563/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:18:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:43.030344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f740ec-b213-49ed-aa30-10c7ed1a31cd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:43.130400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f740ec-b213-49ed-aa30-10c7ed1a31cd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:43.165299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:43.165949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:43.166257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:43.166537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:43.167208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:43.167557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:43.168940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:43.171228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:43.178101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 579/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:18:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:43.184671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02207b21-7381-474d-beaa-825640220fef None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:43.290421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:43.295239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02207b21-7381-474d-beaa-825640220fef None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:43.332169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccf701-42a9-4386-99de-a3cd59a1108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:43.332827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccf701-42a9-4386-99de-a3cd59a1108c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:43.333114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccf701-42a9-4386-99de-a3cd59a1108c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:43.333393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccf701-42a9-4386-99de-a3cd59a1108c None admin] PATH_INFO: `/v3/users/d2274aac7dee443384b002afbeda70a1` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:43.348896 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccf701-42a9-4386-99de-a3cd59a1108c None admin] RBAC: Authorizing `identity:get_user(user_id=d2274aac7dee443384b002afbeda70a1)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:43.350390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccf701-42a9-4386-99de-a3cd59a1108c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:43.352111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 564/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:43 2022] GET /identity/v3/users/d2274aac7dee443384b002afbeda70a1 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:43.358918 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:43.463530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868296b7-8f85-4092-85d1-b899c9d0132f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:43.499895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f37d0-ac0d-4be8-a361-488aec4b051b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:43.500545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f37d0-ac0d-4be8-a361-488aec4b051b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:43.500774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f37d0-ac0d-4be8-a361-488aec4b051b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:43.501054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f37d0-ac0d-4be8-a361-488aec4b051b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:43.566193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:43.570131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f37d0-ac0d-4be8-a361-488aec4b051b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:43.571591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f37d0-ac0d-4be8-a361-488aec4b051b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:43.670572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 580/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:18:43.676650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccf701-42a9-4386-99de-a3cd59a1108c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:43.774507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccf701-42a9-4386-99de-a3cd59a1108c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:43.808546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e10d54-6462-4db4-82ce-7476f8658fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:43.809180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e10d54-6462-4db4-82ce-7476f8658fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:43.809484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e10d54-6462-4db4-82ce-7476f8658fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:43.809768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e10d54-6462-4db4-82ce-7476f8658fa9 None admin] PATH_INFO: `/v3/projects/a91fbb58194a43bea1cb9bdc01dc460b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:43.810885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e10d54-6462-4db4-82ce-7476f8658fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=a91fbb58194a43bea1cb9bdc01dc460b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:43.812327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e10d54-6462-4db4-82ce-7476f8658fa9 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:43.813991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 565/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:43 2022] GET /identity/v3/projects/a91fbb58194a43bea1cb9bdc01dc460b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:43.820218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f37d0-ac0d-4be8-a361-488aec4b051b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:43.914268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:43.918919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f37d0-ac0d-4be8-a361-488aec4b051b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:43.952764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddc3e0-b788-4329-8055-539de17cdaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:43.953371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddc3e0-b788-4329-8055-539de17cdaae None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:43.953679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddc3e0-b788-4329-8055-539de17cdaae None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:43.953953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddc3e0-b788-4329-8055-539de17cdaae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:43.954674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddc3e0-b788-4329-8055-539de17cdaae None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=a91fbb58194a43bea1cb9bdc01dc460b, user.id=d2274aac7dee443384b002afbeda70a1) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:43.954981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddc3e0-b788-4329-8055-539de17cdaae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:43.956403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddc3e0-b788-4329-8055-539de17cdaae None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:43.961800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 581/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:18:43 2022] GET /identity/v3/role_assignments?user.id=d2274aac7dee443384b002afbeda70a1&scope.project.id=a91fbb58194a43bea1cb9bdc01dc460b&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:46.118073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f05f2-d583-4ad5-b688-8a52824e5fd9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:46.118662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f05f2-d583-4ad5-b688-8a52824e5fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:46.118753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f05f2-d583-4ad5-b688-8a52824e5fd9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:46.120176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 566/1147] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:46.126129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1eb608-2515-4030-9466-adee1fc36344 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:46.126885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1eb608-2515-4030-9466-adee1fc36344 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:46.126885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1eb608-2515-4030-9466-adee1fc36344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:46.146904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aa1eb608-2515-4030-9466-adee1fc36344 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:46.148159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aa1eb608-2515-4030-9466-adee1fc36344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:46.223547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 582/1148] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:46.499694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f05f2-d583-4ad5-b688-8a52824e5fd9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:46.526971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:46.611490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f05f2-d583-4ad5-b688-8a52824e5fd9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:46.648536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253d17f-3d1d-4c6f-b349-525c793a0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:46.649166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253d17f-3d1d-4c6f-b349-525c793a0566 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:46.649441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253d17f-3d1d-4c6f-b349-525c793a0566 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:46.649740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253d17f-3d1d-4c6f-b349-525c793a0566 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:46.660455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253d17f-3d1d-4c6f-b349-525c793a0566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8253d17f-3d1d-4c6f-b349-525c793a0566 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:46.665179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:46.666394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:46.667663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 567/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:18:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:46.672448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1eb608-2515-4030-9466-adee1fc36344 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:46.763426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:46.772302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1eb608-2515-4030-9466-adee1fc36344 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:46.805364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:46.805998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:46.806281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:46.806559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:46.807224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:46.807534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:46.809518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:46.817537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 583/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:18:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:46.823451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253d17f-3d1d-4c6f-b349-525c793a0566 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:46.909244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:46.925459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253d17f-3d1d-4c6f-b349-525c793a0566 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:46.958510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849c91a-7876-4266-a1d6-b54f83f916e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:46.959135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849c91a-7876-4266-a1d6-b54f83f916e1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:46.959414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849c91a-7876-4266-a1d6-b54f83f916e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:46.959686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849c91a-7876-4266-a1d6-b54f83f916e1 None admin] PATH_INFO: `/v3/users/d2274aac7dee443384b002afbeda70a1` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:46.960792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849c91a-7876-4266-a1d6-b54f83f916e1 None admin] RBAC: Authorizing `identity:get_user(user_id=d2274aac7dee443384b002afbeda70a1)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:46.962243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849c91a-7876-4266-a1d6-b54f83f916e1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:46.963809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 568/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:46 2022] GET /identity/v3/users/d2274aac7dee443384b002afbeda70a1 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:46.969883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:47.073436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b5924-1e48-4e79-b374-28c5bd39f2e5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:47.109263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04a824-952d-41d4-aa82-e40c90314d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:47.109926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a824-952d-41d4-aa82-e40c90314d2e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:47.110207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a824-952d-41d4-aa82-e40c90314d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:47.110485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a824-952d-41d4-aa82-e40c90314d2e None admin] PATH_INFO: `/v3/projects/a91fbb58194a43bea1cb9bdc01dc460b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:47.111600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04a824-952d-41d4-aa82-e40c90314d2e None admin] RBAC: Authorizing `identity:get_project(project_id=a91fbb58194a43bea1cb9bdc01dc460b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:47.113041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04a824-952d-41d4-aa82-e40c90314d2e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:47.114713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 584/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:46 2022] GET /identity/v3/projects/a91fbb58194a43bea1cb9bdc01dc460b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:47.120903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849c91a-7876-4266-a1d6-b54f83f916e1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:47.219641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849c91a-7876-4266-a1d6-b54f83f916e1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:47.253717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740ce9b-5d8b-46b0-9fbf-560385d623dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:47.254378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740ce9b-5d8b-46b0-9fbf-560385d623dd None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:47.254931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740ce9b-5d8b-46b0-9fbf-560385d623dd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:47.255231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740ce9b-5d8b-46b0-9fbf-560385d623dd None admin] PATH_INFO: `/v3/projects/a91fbb58194a43bea1cb9bdc01dc460b/users/d2274aac7dee443384b002afbeda70a1/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:47.257126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740ce9b-5d8b-46b0-9fbf-560385d623dd None admin] RBAC: Authorizing `identity:create_grant(project_id=a91fbb58194a43bea1cb9bdc01dc460b, user_id=d2274aac7dee443384b002afbeda70a1, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:47.258593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740ce9b-5d8b-46b0-9fbf-560385d623dd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:47.272027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 569/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:18:47 2022] PUT /identity/v3/projects/a91fbb58194a43bea1cb9bdc01dc460b/users/d2274aac7dee443384b002afbeda70a1/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:18:49.446877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b1362-7cb3-428c-8a02-62549fd46498 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:49.447431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b1362-7cb3-428c-8a02-62549fd46498 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:49.447431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b1362-7cb3-428c-8a02-62549fd46498 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:49.449319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 585/1154] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:49.454752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bf50a-1609-46a2-a0ea-c9dc6471ecad None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:49.455187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bf50a-1609-46a2-a0ea-c9dc6471ecad None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:49.455313 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bf50a-1609-46a2-a0ea-c9dc6471ecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:49.474176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b32bf50a-1609-46a2-a0ea-c9dc6471ecad None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:49.475399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b32bf50a-1609-46a2-a0ea-c9dc6471ecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:49.483116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:49.545298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 570/1155] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:49.818373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b1362-7cb3-428c-8a02-62549fd46498 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:49.844576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:49.930346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b1362-7cb3-428c-8a02-62549fd46498 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:49.968131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c1723-01f2-49d7-b981-419833e17199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:49.968775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c1723-01f2-49d7-b981-419833e17199 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:49.969057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c1723-01f2-49d7-b981-419833e17199 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:49.969331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c1723-01f2-49d7-b981-419833e17199 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:49.981397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c1723-01f2-49d7-b981-419833e17199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-844c1723-01f2-49d7-b981-419833e17199 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:18:49.986304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:49.987552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:18:49.988481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 586/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:18:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:49.993870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bf50a-1609-46a2-a0ea-c9dc6471ecad None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:50.094779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bf50a-1609-46a2-a0ea-c9dc6471ecad None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:50.129197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:50.130105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:50.130385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:50.130660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:50.131313 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:50.131617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:50.132990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:50.140979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 571/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:18:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:50.146989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c1723-01f2-49d7-b981-419833e17199 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:50.227538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:50.246958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c1723-01f2-49d7-b981-419833e17199 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:50.280388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85586ad9-aa34-4763-b3b8-e4d6f6a36914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:50.281016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85586ad9-aa34-4763-b3b8-e4d6f6a36914 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:50.281302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85586ad9-aa34-4763-b3b8-e4d6f6a36914 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:50.281689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85586ad9-aa34-4763-b3b8-e4d6f6a36914 None admin] PATH_INFO: `/v3/users/d2274aac7dee443384b002afbeda70a1` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:50.282867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85586ad9-aa34-4763-b3b8-e4d6f6a36914 None admin] RBAC: Authorizing `identity:get_user(user_id=d2274aac7dee443384b002afbeda70a1)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:50.284325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85586ad9-aa34-4763-b3b8-e4d6f6a36914 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:50.285959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 587/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:18:50 2022] GET /identity/v3/users/d2274aac7dee443384b002afbeda70a1 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:50.291801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:50.296376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:50.391133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25275ff-316a-406a-995a-86f9597e5124 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:50.424480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e48b8-df63-43ba-b23b-14b569b77798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:50.425092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e48b8-df63-43ba-b23b-14b569b77798 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:50.425376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e48b8-df63-43ba-b23b-14b569b77798 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:50.425678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e48b8-df63-43ba-b23b-14b569b77798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:50.491683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e48b8-df63-43ba-b23b-14b569b77798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:50.493150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e48b8-df63-43ba-b23b-14b569b77798 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:50.496317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:50.591463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 572/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 13:18:50.597647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85586ad9-aa34-4763-b3b8-e4d6f6a36914 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:50.703350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85586ad9-aa34-4763-b3b8-e4d6f6a36914 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:50.741251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f3813-be1e-446e-84df-dd3fc39406f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:50.741921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3813-be1e-446e-84df-dd3fc39406f2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:50.742209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3813-be1e-446e-84df-dd3fc39406f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:50.742488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f3813-be1e-446e-84df-dd3fc39406f2 None admin] PATH_INFO: `/v3/projects/a91fbb58194a43bea1cb9bdc01dc460b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:50.743619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f3813-be1e-446e-84df-dd3fc39406f2 None admin] RBAC: Authorizing `identity:get_project(project_id=a91fbb58194a43bea1cb9bdc01dc460b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:50.745062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f3813-be1e-446e-84df-dd3fc39406f2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:50.746880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 588/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:18:50 2022] GET /identity/v3/projects/a91fbb58194a43bea1cb9bdc01dc460b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:50.753114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e48b8-df63-43ba-b23b-14b569b77798 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:50.853788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e48b8-df63-43ba-b23b-14b569b77798 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:50.856119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:50.888530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572747b-09d5-487f-aee9-abb80999a635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:50.889134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572747b-09d5-487f-aee9-abb80999a635 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:50.889418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572747b-09d5-487f-aee9-abb80999a635 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:50.889729 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572747b-09d5-487f-aee9-abb80999a635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:50.890455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572747b-09d5-487f-aee9-abb80999a635 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=a91fbb58194a43bea1cb9bdc01dc460b, user.id=d2274aac7dee443384b002afbeda70a1) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:50.890789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572747b-09d5-487f-aee9-abb80999a635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:50.892198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572747b-09d5-487f-aee9-abb80999a635 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:50.897990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 573/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:18:50 2022] GET /identity/v3/role_assignments?user.id=d2274aac7dee443384b002afbeda70a1&scope.project.id=a91fbb58194a43bea1cb9bdc01dc460b&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:53.099099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23b535-2161-407e-8bd4-ef269e210d64 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:53.099640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23b535-2161-407e-8bd4-ef269e210d64 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:53.100021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23b535-2161-407e-8bd4-ef269e210d64 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:53.101378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 589/1162] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:53.108076 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9c60-25e1-44d7-a210-60c1a51b646d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:53.108617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9c60-25e1-44d7-a210-60c1a51b646d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:53.108617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f9c60-25e1-44d7-a210-60c1a51b646d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:53.127495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a72f9c60-25e1-44d7-a210-60c1a51b646d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:53.128725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a72f9c60-25e1-44d7-a210-60c1a51b646d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:53.197472 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 574/1163] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:53.470934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23b535-2161-407e-8bd4-ef269e210d64 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:53.491068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:53.575779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23b535-2161-407e-8bd4-ef269e210d64 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:53.610358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ef34c-642a-4063-8e3d-6e4654cab1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:53.610975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ef34c-642a-4063-8e3d-6e4654cab1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:53.611260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ef34c-642a-4063-8e3d-6e4654cab1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:53.611533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ef34c-642a-4063-8e3d-6e4654cab1d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:53.677999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ef34c-642a-4063-8e3d-6e4654cab1d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:53.679444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ef34c-642a-4063-8e3d-6e4654cab1d6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:53.696882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:53.779085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 590/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:18:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:18:53.784947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f9c60-25e1-44d7-a210-60c1a51b646d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:53.884035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:53.888752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f9c60-25e1-44d7-a210-60c1a51b646d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:53.922728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197f8a-7c8a-4aa2-bf03-3a7cb724c92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:53.923368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197f8a-7c8a-4aa2-bf03-3a7cb724c92c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:53.923653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197f8a-7c8a-4aa2-bf03-3a7cb724c92c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:53.923927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197f8a-7c8a-4aa2-bf03-3a7cb724c92c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:53.930651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197f8a-7c8a-4aa2-bf03-3a7cb724c92c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:53.932094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197f8a-7c8a-4aa2-bf03-3a7cb724c92c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-30197f8a-7c8a-4aa2-bf03-3a7cb724c92c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:18:53.944299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:18:53.945596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:18:53.946486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 575/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:18:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:18:53.952332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ef34c-642a-4063-8e3d-6e4654cab1d6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:54.049776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ef34c-642a-4063-8e3d-6e4654cab1d6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:54.066901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:54.084315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8d515-a37e-4799-8802-f7900ca59e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:54.084926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8d515-a37e-4799-8802-f7900ca59e50 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:54.085212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8d515-a37e-4799-8802-f7900ca59e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:54.085515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8d515-a37e-4799-8802-f7900ca59e50 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:54.086164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8d515-a37e-4799-8802-f7900ca59e50 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:18:54.086473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8d515-a37e-4799-8802-f7900ca59e50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:54.087862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b8d515-a37e-4799-8802-f7900ca59e50 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:54.094995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 591/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Mon Oct 3 13:18:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:56.287717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5721-78d9-41f0-9666-9b4c9c3313bb None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:56.288255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5721-78d9-41f0-9666-9b4c9c3313bb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:56.288255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a5721-78d9-41f0-9666-9b4c9c3313bb None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:56.289656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 576/1167] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:56.295930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f391e-09e9-4564-a784-b78a51f9c7ce None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:56.296380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f391e-09e9-4564-a784-b78a51f9c7ce None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:56.296482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f391e-09e9-4564-a784-b78a51f9c7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:56.315647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9e8f391e-09e9-4564-a784-b78a51f9c7ce None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:56.316872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9e8f391e-09e9-4564-a784-b78a51f9c7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:56.386160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 592/1168] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:56.660503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a5721-78d9-41f0-9666-9b4c9c3313bb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:56.763710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a5721-78d9-41f0-9666-9b4c9c3313bb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:56.797886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34faf4-9958-432d-9f2b-db7cfe08311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:56.798505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34faf4-9958-432d-9f2b-db7cfe08311a None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:56.798789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34faf4-9958-432d-9f2b-db7cfe08311a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:56.799065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34faf4-9958-432d-9f2b-db7cfe08311a None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:56.799733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34faf4-9958-432d-9f2b-db7cfe08311a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:56.801101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34faf4-9958-432d-9f2b-db7cfe08311a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:56.817250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 577/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Mon Oct 3 13:18:56 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:18:59.093954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a276a7c-8780-47e5-9c30-a28fa6ac145e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:59.094503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a276a7c-8780-47e5-9c30-a28fa6ac145e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:59.094503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a276a7c-8780-47e5-9c30-a28fa6ac145e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:59.095874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 593/1170] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:18:59.102404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889913bc-f9a8-45bb-802c-c7be4cd04ab5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:59.102920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889913bc-f9a8-45bb-802c-c7be4cd04ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:59.103137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889913bc-f9a8-45bb-802c-c7be4cd04ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:59.129195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-889913bc-f9a8-45bb-802c-c7be4cd04ab5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:18:59.130733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-889913bc-f9a8-45bb-802c-c7be4cd04ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:18:59.146919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:59.212163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 578/1171] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:18:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:18:59.548661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a276a7c-8780-47e5-9c30-a28fa6ac145e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:59.668412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a276a7c-8780-47e5-9c30-a28fa6ac145e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:59.683134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:59.714508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd580e-5619-42f0-8ce7-5b6d568ab616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:59.715606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd580e-5619-42f0-8ce7-5b6d568ab616 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:59.716129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd580e-5619-42f0-8ce7-5b6d568ab616 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:59.716632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd580e-5619-42f0-8ce7-5b6d568ab616 None admin] PATH_INFO: `/v3/domains/8e191242a8f64b748a3f14b89baea440` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:59.729693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd580e-5619-42f0-8ce7-5b6d568ab616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8e191242a8f64b748a3f14b89baea440)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:59.731619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd580e-5619-42f0-8ce7-5b6d568ab616 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:59.733281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 594/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Mon Oct 3 13:18:59 2022] GET /identity/v3/domains/8e191242a8f64b748a3f14b89baea440 => generated 267 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:18:59.739226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889913bc-f9a8-45bb-802c-c7be4cd04ab5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:18:59.857965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889913bc-f9a8-45bb-802c-c7be4cd04ab5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:18:59.898981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315349d-b34d-4d00-9a41-0d51cfbca6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:18:59.899648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315349d-b34d-4d00-9a41-0d51cfbca6e3 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:18:59.900019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315349d-b34d-4d00-9a41-0d51cfbca6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:18:59.900317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315349d-b34d-4d00-9a41-0d51cfbca6e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:18:59.901077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315349d-b34d-4d00-9a41-0d51cfbca6e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:18:59.902760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315349d-b34d-4d00-9a41-0d51cfbca6e3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:18:59.927758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:18:59.936269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 579/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:18:59 2022] POST /identity/v3/projects => generated 361 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:19:02.201886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6dde0-9256-4b26-80d8-b0c9de4d68c0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:02.202555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6dde0-9256-4b26-80d8-b0c9de4d68c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:02.202555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6dde0-9256-4b26-80d8-b0c9de4d68c0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:02.203810 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 595/1174] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:19:02.209538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7b89-69bf-43cf-9d8e-82234205778d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:02.210032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7b89-69bf-43cf-9d8e-82234205778d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:02.210125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7b89-69bf-43cf-9d8e-82234205778d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:02.230732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d93e7b89-69bf-43cf-9d8e-82234205778d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:19:02.231960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d93e7b89-69bf-43cf-9d8e-82234205778d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:19:02.309787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 580/1175] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:19:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:19:02.610508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6dde0-9256-4b26-80d8-b0c9de4d68c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:02.728367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6dde0-9256-4b26-80d8-b0c9de4d68c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:02.769968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4caf3-5d89-4792-b88f-d64613ad2622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:02.770631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4caf3-5d89-4792-b88f-d64613ad2622 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:02.770921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4caf3-5d89-4792-b88f-d64613ad2622 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:02.771202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4caf3-5d89-4792-b88f-d64613ad2622 None admin] PATH_INFO: `/v3/domains/8e191242a8f64b748a3f14b89baea440` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:02.772439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4caf3-5d89-4792-b88f-d64613ad2622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8e191242a8f64b748a3f14b89baea440)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:02.773963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4caf3-5d89-4792-b88f-d64613ad2622 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:02.775650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 596/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Mon Oct 3 13:19:02 2022] GET /identity/v3/domains/8e191242a8f64b748a3f14b89baea440 => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:02.782208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7b89-69bf-43cf-9d8e-82234205778d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:02.893753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7b89-69bf-43cf-9d8e-82234205778d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:02.927131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:02.934055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b08c1-81e9-4607-9f25-f8ac87962237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:02.934692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b08c1-81e9-4607-9f25-f8ac87962237 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:02.934974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b08c1-81e9-4607-9f25-f8ac87962237 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:02.935246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b08c1-81e9-4607-9f25-f8ac87962237 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:02.935910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b08c1-81e9-4607-9f25-f8ac87962237 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:02.937371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b08c1-81e9-4607-9f25-f8ac87962237 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:02.960237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 581/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:19:02 2022] POST /identity/v3/users => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:19:05.252727 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0891433-0fce-49ba-ba7c-8805f85890ad None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:05.253282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0891433-0fce-49ba-ba7c-8805f85890ad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:05.253282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0891433-0fce-49ba-ba7c-8805f85890ad None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:05.254953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 597/1178] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:19:05.260913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2781e9a-fd39-41f6-bfb0-a18bf3bd3cfe None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:05.261518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2781e9a-fd39-41f6-bfb0-a18bf3bd3cfe None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:05.261518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2781e9a-fd39-41f6-bfb0-a18bf3bd3cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:05.282110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a2781e9a-fd39-41f6-bfb0-a18bf3bd3cfe None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:19:05.283592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a2781e9a-fd39-41f6-bfb0-a18bf3bd3cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:19:05.364129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 582/1179] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:19:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:19:05.651950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0891433-0fce-49ba-ba7c-8805f85890ad None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:05.700454 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:05.767122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0891433-0fce-49ba-ba7c-8805f85890ad None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:05.803729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1655d0c-149e-43b8-81b0-a9b1f906078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:05.804431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1655d0c-149e-43b8-81b0-a9b1f906078d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:05.804640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1655d0c-149e-43b8-81b0-a9b1f906078d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:05.804915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1655d0c-149e-43b8-81b0-a9b1f906078d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:05.816665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1655d0c-149e-43b8-81b0-a9b1f906078d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c1655d0c-149e-43b8-81b0-a9b1f906078d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:19:05.821402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:19:05.822536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:19:05.823545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 598/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:19:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:19:05.829479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2781e9a-fd39-41f6-bfb0-a18bf3bd3cfe None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:05.948495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2781e9a-fd39-41f6-bfb0-a18bf3bd3cfe None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:05.971413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:06.001754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:06.002858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:06.003349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:06.003858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:06.005348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:19:06.005844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:06.008474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:06.019214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 583/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:19:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:06.026629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1655d0c-149e-43b8-81b0-a9b1f906078d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:06.142211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:06.147316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1655d0c-149e-43b8-81b0-a9b1f906078d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:06.192961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e827a-0eaa-4f8b-9fd8-8b0919293dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:06.193706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e827a-0eaa-4f8b-9fd8-8b0919293dae None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:06.194131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e827a-0eaa-4f8b-9fd8-8b0919293dae None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:06.194548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e827a-0eaa-4f8b-9fd8-8b0919293dae None admin] PATH_INFO: `/v3/users/f6b6637bc8784072a0051bf39036e49c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:06.211225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e827a-0eaa-4f8b-9fd8-8b0919293dae None admin] RBAC: Authorizing `identity:get_user(user_id=f6b6637bc8784072a0051bf39036e49c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:06.212639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e827a-0eaa-4f8b-9fd8-8b0919293dae None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:06.215653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 599/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:19:06 2022] GET /identity/v3/users/f6b6637bc8784072a0051bf39036e49c => generated 311 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:06.222246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:06.334471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e969c-1dae-4c65-aca4-c9e8b428f2c1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:06.387936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25d96-5faa-459d-89ac-bde22361757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:06.388555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25d96-5faa-459d-89ac-bde22361757e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:06.388843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25d96-5faa-459d-89ac-bde22361757e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:06.389121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b25d96-5faa-459d-89ac-bde22361757e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:06.445346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:06.471956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b25d96-5faa-459d-89ac-bde22361757e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:06.473410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b25d96-5faa-459d-89ac-bde22361757e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:06.578570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 584/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:19:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:19:06.584723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e827a-0eaa-4f8b-9fd8-8b0919293dae None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:06.700078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e827a-0eaa-4f8b-9fd8-8b0919293dae None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:06.740437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d3b4eb-629f-45e1-ba6f-ba08ed849c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:06.740992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3b4eb-629f-45e1-ba6f-ba08ed849c7a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:06.741274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3b4eb-629f-45e1-ba6f-ba08ed849c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:06.741582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3b4eb-629f-45e1-ba6f-ba08ed849c7a None admin] PATH_INFO: `/v3/projects/27f5a383303e478ba7516f22fc122e63` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:06.742714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3b4eb-629f-45e1-ba6f-ba08ed849c7a None admin] RBAC: Authorizing `identity:get_project(project_id=27f5a383303e478ba7516f22fc122e63)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:06.744152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d3b4eb-629f-45e1-ba6f-ba08ed849c7a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:06.746429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 600/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:19:06 2022] GET /identity/v3/projects/27f5a383303e478ba7516f22fc122e63 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:06.752967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25d96-5faa-459d-89ac-bde22361757e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:06.862289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:06.886407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b25d96-5faa-459d-89ac-bde22361757e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:06.924115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b856ce9-7565-4507-b19e-aa97a77db250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:06.924719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856ce9-7565-4507-b19e-aa97a77db250 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:06.924998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856ce9-7565-4507-b19e-aa97a77db250 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:06.925270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b856ce9-7565-4507-b19e-aa97a77db250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:06.926024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b856ce9-7565-4507-b19e-aa97a77db250 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=27f5a383303e478ba7516f22fc122e63, user.id=f6b6637bc8784072a0051bf39036e49c) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:19:06.926329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b856ce9-7565-4507-b19e-aa97a77db250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:06.927878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b856ce9-7565-4507-b19e-aa97a77db250 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:06.933573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 585/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:19:06 2022] GET /identity/v3/role_assignments?user.id=f6b6637bc8784072a0051bf39036e49c&scope.project.id=27f5a383303e478ba7516f22fc122e63&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:19:09.132832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893bde-a963-4e03-b6aa-769d9a3fba7e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:09.133487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893bde-a963-4e03-b6aa-769d9a3fba7e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:09.133487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893bde-a963-4e03-b6aa-769d9a3fba7e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:09.134752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 601/1186] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:19:09.140538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172a8d-233a-42e7-b684-769707d0c295 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:09.141110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172a8d-233a-42e7-b684-769707d0c295 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:09.141110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98172a8d-233a-42e7-b684-769707d0c295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:09.161924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-98172a8d-233a-42e7-b684-769707d0c295 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:19:09.163177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-98172a8d-233a-42e7-b684-769707d0c295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:19:09.233949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 586/1187] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:19:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:19:09.500013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b893bde-a963-4e03-b6aa-769d9a3fba7e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:09.527868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:09.603384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b893bde-a963-4e03-b6aa-769d9a3fba7e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:09.637189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53844d6b-30bf-4363-b488-f67786c7dd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:09.637963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53844d6b-30bf-4363-b488-f67786c7dd84 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:09.638250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53844d6b-30bf-4363-b488-f67786c7dd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:09.638527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53844d6b-30bf-4363-b488-f67786c7dd84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:09.648282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53844d6b-30bf-4363-b488-f67786c7dd84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-53844d6b-30bf-4363-b488-f67786c7dd84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:19:09.653310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:19:09.654509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:19:09.655601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 602/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:19:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:19:09.660573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98172a8d-233a-42e7-b684-769707d0c295 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:09.729779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:09.759621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98172a8d-233a-42e7-b684-769707d0c295 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:09.793823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:09.794432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:09.794710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:09.794980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:09.795638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:19:09.795940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:09.797333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:09.805170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 587/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:19:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:09.811289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53844d6b-30bf-4363-b488-f67786c7dd84 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:09.909044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:09.928328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53844d6b-30bf-4363-b488-f67786c7dd84 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:09.969016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115f383-1444-4682-af37-74721e713440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:09.969680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115f383-1444-4682-af37-74721e713440 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:09.969965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115f383-1444-4682-af37-74721e713440 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:09.970244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115f383-1444-4682-af37-74721e713440 None admin] PATH_INFO: `/v3/users/f6b6637bc8784072a0051bf39036e49c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:09.971367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115f383-1444-4682-af37-74721e713440 None admin] RBAC: Authorizing `identity:get_user(user_id=f6b6637bc8784072a0051bf39036e49c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:09.972806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115f383-1444-4682-af37-74721e713440 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:09.974432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 603/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:19:09 2022] GET /identity/v3/users/f6b6637bc8784072a0051bf39036e49c => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:09.980940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:10.080046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c043f-90e0-4cbd-a296-624f5ac99855 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:10.113701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f2291-0179-4726-b9c7-1c332d9b50cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:10.114334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f2291-0179-4726-b9c7-1c332d9b50cd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:10.114618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f2291-0179-4726-b9c7-1c332d9b50cd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:10.114895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f2291-0179-4726-b9c7-1c332d9b50cd None admin] PATH_INFO: `/v3/projects/27f5a383303e478ba7516f22fc122e63` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:10.116001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f2291-0179-4726-b9c7-1c332d9b50cd None admin] RBAC: Authorizing `identity:get_project(project_id=27f5a383303e478ba7516f22fc122e63)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:10.117435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90f2291-0179-4726-b9c7-1c332d9b50cd None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:10.119101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 588/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:19:09 2022] GET /identity/v3/projects/27f5a383303e478ba7516f22fc122e63 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:10.125178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115f383-1444-4682-af37-74721e713440 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:10.223710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115f383-1444-4682-af37-74721e713440 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:10.256465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cefe6f-3227-4b11-9ab0-d340b065718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:10.257114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cefe6f-3227-4b11-9ab0-d340b065718c None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:10.257397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cefe6f-3227-4b11-9ab0-d340b065718c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:10.257702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cefe6f-3227-4b11-9ab0-d340b065718c None admin] PATH_INFO: `/v3/projects/27f5a383303e478ba7516f22fc122e63/users/f6b6637bc8784072a0051bf39036e49c/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:10.259571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cefe6f-3227-4b11-9ab0-d340b065718c None admin] RBAC: Authorizing `identity:create_grant(project_id=27f5a383303e478ba7516f22fc122e63, user_id=f6b6637bc8784072a0051bf39036e49c, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:10.261597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cefe6f-3227-4b11-9ab0-d340b065718c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:10.274387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 604/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:19:10 2022] PUT /identity/v3/projects/27f5a383303e478ba7516f22fc122e63/users/f6b6637bc8784072a0051bf39036e49c/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:19:12.470555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde9076-8849-4f62-b7b7-f8dbe6962caf None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:12.471104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde9076-8849-4f62-b7b7-f8dbe6962caf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:12.471104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde9076-8849-4f62-b7b7-f8dbe6962caf None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:12.472458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 589/1193] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:19:12.477651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88d4f-4aa2-4609-acc3-3dc46d014aff None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:12.478042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88d4f-4aa2-4609-acc3-3dc46d014aff None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:12.478209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88d4f-4aa2-4609-acc3-3dc46d014aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:12.497480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aba88d4f-4aa2-4609-acc3-3dc46d014aff None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:19:12.498715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aba88d4f-4aa2-4609-acc3-3dc46d014aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:19:12.508339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:12.567021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 605/1194] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:19:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:19:12.837816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde9076-8849-4f62-b7b7-f8dbe6962caf None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:12.841253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:12.945284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde9076-8849-4f62-b7b7-f8dbe6962caf None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:12.979993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5a68-88bb-47e4-ba71-b6526ccb72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:12.980634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5a68-88bb-47e4-ba71-b6526ccb72d8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:12.980917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5a68-88bb-47e4-ba71-b6526ccb72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:12.981195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5a68-88bb-47e4-ba71-b6526ccb72d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:12.992108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a5a68-88bb-47e4-ba71-b6526ccb72d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-824a5a68-88bb-47e4-ba71-b6526ccb72d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:19:12.997734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:19:13.001491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:19:13.002654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 590/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:19:13.005292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba88d4f-4aa2-4609-acc3-3dc46d014aff None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:13.102267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba88d4f-4aa2-4609-acc3-3dc46d014aff None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:13.139400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:13.140027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:13.140307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:13.140582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:13.141265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:19:13.141602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:13.143001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:13.152337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 606/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:19:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:13.158414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5a68-88bb-47e4-ba71-b6526ccb72d8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:13.219672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:13.266439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5a68-88bb-47e4-ba71-b6526ccb72d8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:13.300325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dde075-202a-42da-869e-8e83024b1347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:13.300966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dde075-202a-42da-869e-8e83024b1347 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:13.301254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dde075-202a-42da-869e-8e83024b1347 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:13.301559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dde075-202a-42da-869e-8e83024b1347 None admin] PATH_INFO: `/v3/users/f6b6637bc8784072a0051bf39036e49c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:13.302701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dde075-202a-42da-869e-8e83024b1347 None admin] RBAC: Authorizing `identity:get_user(user_id=f6b6637bc8784072a0051bf39036e49c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:13.304142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dde075-202a-42da-869e-8e83024b1347 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:13.305769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 591/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Mon Oct 3 13:19:13 2022] GET /identity/v3/users/f6b6637bc8784072a0051bf39036e49c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:19:13.311675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:13.320927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:13.411881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e88ef-4d28-4489-892c-7e7887fc897a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:13.445246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f536b3-eba3-4c69-916b-b05a29732ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:13.445877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f536b3-eba3-4c69-916b-b05a29732ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:13.446159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f536b3-eba3-4c69-916b-b05a29732ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:13.446432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f536b3-eba3-4c69-916b-b05a29732ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:13.512177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f536b3-eba3-4c69-916b-b05a29732ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:13.513663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f536b3-eba3-4c69-916b-b05a29732ec7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:13.521266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:13.609750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 607/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:19:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:19:13.615789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dde075-202a-42da-869e-8e83024b1347 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:13.716953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dde075-202a-42da-869e-8e83024b1347 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:13.746133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:13.752256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9533190-0e24-4d06-96de-1395961cba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:13.752893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9533190-0e24-4d06-96de-1395961cba0a None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:13.753178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9533190-0e24-4d06-96de-1395961cba0a None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:13.753468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9533190-0e24-4d06-96de-1395961cba0a None admin] PATH_INFO: `/v3/projects/27f5a383303e478ba7516f22fc122e63` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:13.754867 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9533190-0e24-4d06-96de-1395961cba0a None admin] RBAC: Authorizing `identity:get_project(project_id=27f5a383303e478ba7516f22fc122e63)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:13.756301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9533190-0e24-4d06-96de-1395961cba0a None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:13.757997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 592/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Mon Oct 3 13:19:13 2022] GET /identity/v3/projects/27f5a383303e478ba7516f22fc122e63 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:13.764824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f536b3-eba3-4c69-916b-b05a29732ec7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:19:13.871423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f536b3-eba3-4c69-916b-b05a29732ec7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:19:13.880221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:13.910518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd8915-23a0-40c7-a9ff-41201d6d828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:19:13.911144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8915-23a0-40c7-a9ff-41201d6d828e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:19:13.911423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8915-23a0-40c7-a9ff-41201d6d828e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:19:13.911695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd8915-23a0-40c7-a9ff-41201d6d828e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:19:13.912440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd8915-23a0-40c7-a9ff-41201d6d828e None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=27f5a383303e478ba7516f22fc122e63, user.id=f6b6637bc8784072a0051bf39036e49c) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:19:13.912750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd8915-23a0-40c7-a9ff-41201d6d828e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:19:13.914251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd8915-23a0-40c7-a9ff-41201d6d828e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:19:13.921087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 608/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:19:13 2022] GET /identity/v3/role_assignments?user.id=f6b6637bc8784072a0051bf39036e49c&scope.project.id=27f5a383303e478ba7516f22fc122e63&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:19:23.749862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:19:23.918116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=15 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:13.496787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf19c-9ec1-4a1f-bea6-e78a8df73f51 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:13.497409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf19c-9ec1-4a1f-bea6-e78a8df73f51 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:13.497888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebf19c-9ec1-4a1f-bea6-e78a8df73f51 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:13.500626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 593/1201] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:13.511853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc0a9-2372-4b4f-9094-31c4092e9c92 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:13.511853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc0a9-2372-4b4f-9094-31c4092e9c92 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:13.511853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc0a9-2372-4b4f-9094-31c4092e9c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:13.555715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ad9fc0a9-2372-4b4f-9094-31c4092e9c92 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:13.558985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ad9fc0a9-2372-4b4f-9094-31c4092e9c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:13.677607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 609/1202] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:20:14.043905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebf19c-9ec1-4a1f-bea6-e78a8df73f51 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:14.205166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebf19c-9ec1-4a1f-bea6-e78a8df73f51 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:14.255915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:14.256709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:14.257025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:14.257407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:14.284256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:14.286493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:14.314795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:14.316090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:14.317252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:14.317252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:14.317252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:14.317252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 594/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:14.325869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fc0a9-2372-4b4f-9094-31c4092e9c92 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:14.490445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fc0a9-2372-4b4f-9094-31c4092e9c92 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:14.544849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:14.566480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d11cd3-6d6f-46ff-b189-1be524fc56d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:14.567098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11cd3-6d6f-46ff-b189-1be524fc56d1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:14.567590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11cd3-6d6f-46ff-b189-1be524fc56d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:14.567979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d11cd3-6d6f-46ff-b189-1be524fc56d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:14.569400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d11cd3-6d6f-46ff-b189-1be524fc56d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:14.570681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d11cd3-6d6f-46ff-b189-1be524fc56d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:14.573642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d11cd3-6d6f-46ff-b189-1be524fc56d1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:14.593485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 610/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:20:14.596946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:14.677964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:14.766090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9b822-a93c-420a-9507-d5a3a39ff932 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:14.822601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5a5bd-7afd-464b-b6dc-efe75a8bc69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:14.823364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5a5bd-7afd-464b-b6dc-efe75a8bc69d None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:14.823706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5a5bd-7afd-464b-b6dc-efe75a8bc69d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:14.823976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5a5bd-7afd-464b-b6dc-efe75a8bc69d None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:14.825487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5a5bd-7afd-464b-b6dc-efe75a8bc69d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:14.826573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5a5bd-7afd-464b-b6dc-efe75a8bc69d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:14.856591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 595/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Mon Oct 3 13:20:14 2022] POST /identity/v3/users => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:20:17.694982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7c89a-a375-4622-a7a3-7de5d5582fdb None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:17.695585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7c89a-a375-4622-a7a3-7de5d5582fdb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:17.695585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7c89a-a375-4622-a7a3-7de5d5582fdb None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:17.696884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 611/1206] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:17.702923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9090-e1f9-4085-9c6b-ad94ab883b8f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:17.704581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9090-e1f9-4085-9c6b-ad94ab883b8f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:17.704581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d9090-e1f9-4085-9c6b-ad94ab883b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:17.724060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c95d9090-e1f9-4085-9c6b-ad94ab883b8f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:17.726401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c95d9090-e1f9-4085-9c6b-ad94ab883b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:17.834482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 596/1207] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:20:18.134830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7c89a-a375-4622-a7a3-7de5d5582fdb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:18.298264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f7c89a-a375-4622-a7a3-7de5d5582fdb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:18.359437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01661fef-af9c-4f72-9055-0ae88aa9c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:18.360343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01661fef-af9c-4f72-9055-0ae88aa9c496 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:18.360728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01661fef-af9c-4f72-9055-0ae88aa9c496 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:18.361104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01661fef-af9c-4f72-9055-0ae88aa9c496 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:18.374668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01661fef-af9c-4f72-9055-0ae88aa9c496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-01661fef-af9c-4f72-9055-0ae88aa9c496 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:20:18.379845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:18.380838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:18.382575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 612/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:20:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:18.391330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d9090-e1f9-4085-9c6b-ad94ab883b8f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:18.452040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:18.541080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d9090-e1f9-4085-9c6b-ad94ab883b8f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:18.605282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:18.606208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:18.606664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:18.607105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:18.607958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:18.608409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:18.610042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:18.626324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 597/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:20:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:18.627758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01661fef-af9c-4f72-9055-0ae88aa9c496 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:18.686074 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:18.744755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01661fef-af9c-4f72-9055-0ae88aa9c496 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:18.784420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:18.785072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:18.785355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:18.785669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:18.797254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:18.798760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:18.815991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:18.817217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:18.818533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:18.818533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:18.818533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:18.821687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 613/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:18.823744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:18.959024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:18.963590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdadcf7-2b1f-46c8-ab7e-4fac528db9d6 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:19.006097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:19.007033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:19.007481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:19.007945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:19.008848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:19.009328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:19.011086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:19.026194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 598/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:19.032040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:19.151147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e760c-8b6c-45df-a2d4-1ee555a3c253 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:19.152641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:19.213368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:19.213368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:19.213368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:19.213368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:19.224944 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:19.227328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:20:19.242323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:19.244225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:19.245296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 614/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:20:19.250783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:19.377758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad12e-94be-469b-a907-e174779dbae7 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:19.421522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:19.422284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:19.422610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:19.422928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:19.423748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:19.424102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:19.425714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:19.453428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 599/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Oct 3 13:20:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:19.460100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:19.583150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3522cf-d9ac-4ca4-93ae-521be966e762 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:19.629189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd5ebd-9002-40c8-9fa5-a97af904c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:19.630032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd5ebd-9002-40c8-9fa5-a97af904c075 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:19.630413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd5ebd-9002-40c8-9fa5-a97af904c075 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:19.630894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd5ebd-9002-40c8-9fa5-a97af904c075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:19.668513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:19.709685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd5ebd-9002-40c8-9fa5-a97af904c075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:19.712050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd5ebd-9002-40c8-9fa5-a97af904c075 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:19.829724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 615/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:20:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:20:19.837525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:19.857587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:19.969494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b2273-858c-44fa-b806-67f8b72032f4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:20.009684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:20.010328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:20.010605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:20.010880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:20.021582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:20.025385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:20.043984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:20.051004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:20.052120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:20.052120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:20.052120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:20.052120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 600/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:20.052404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd5ebd-9002-40c8-9fa5-a97af904c075 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:20.118504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:20.161560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd5ebd-9002-40c8-9fa5-a97af904c075 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:20.200718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:20.201363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:20.201712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:20.202131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:20.202705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:20.203022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:20.204512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:20.218078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 616/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:20.223847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:20.318300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:20.360871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7ba2-cf40-4121-b47c-3b2d48766e5d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:20.407465 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:20.407964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:20.408247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:20.408521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:20.416829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:20.418332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:20:20.433185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:20.435487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:20.436519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 601/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:20:20.443203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:20.562889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd28cb-4f0c-4358-88f1-730bea85be5a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:20.587404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:20.599662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb10f7-2546-411c-bfea-0595efb62c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:20.600271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb10f7-2546-411c-bfea-0595efb62c96 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:20.600556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb10f7-2546-411c-bfea-0595efb62c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:20.600915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb10f7-2546-411c-bfea-0595efb62c96 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:20.601651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb10f7-2546-411c-bfea-0595efb62c96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:20.601987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb10f7-2546-411c-bfea-0595efb62c96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:20.603425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb10f7-2546-411c-bfea-0595efb62c96 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:20.615288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 617/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:20.622527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:20.735721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9151a22-573b-4d69-8a5e-3d9b39525b9d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:20.767943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:20.774501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67104642-bcdd-48fc-ad78-343ca56359a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:20.775300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67104642-bcdd-48fc-ad78-343ca56359a4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:20.775686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67104642-bcdd-48fc-ad78-343ca56359a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:20.776072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67104642-bcdd-48fc-ad78-343ca56359a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:20.776993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67104642-bcdd-48fc-ad78-343ca56359a4 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=7bca29b8758b408da2bf05face26b551) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:20.777334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67104642-bcdd-48fc-ad78-343ca56359a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:20.778959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67104642-bcdd-48fc-ad78-343ca56359a4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:20.785342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 602/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:20:20 2022] GET /identity/v3/role_assignments?user.id=7bca29b8758b408da2bf05face26b551&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:23.031182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053169f-cde8-4598-b944-46889ab7e692 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:23.032001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053169f-cde8-4598-b944-46889ab7e692 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:23.032001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053169f-cde8-4598-b944-46889ab7e692 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:23.033427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 618/1220] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:20:23.040139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7895-9bc5-4be8-b799-3012d90bb2fc None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:23.040664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7895-9bc5-4be8-b799-3012d90bb2fc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:23.040831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f7895-9bc5-4be8-b799-3012d90bb2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:23.063948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-602f7895-9bc5-4be8-b799-3012d90bb2fc None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:23.065557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-602f7895-9bc5-4be8-b799-3012d90bb2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:23.144053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 603/1221] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:20:23.427833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053169f-cde8-4598-b944-46889ab7e692 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:23.555362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053169f-cde8-4598-b944-46889ab7e692 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:23.594251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aa708-fcf7-42fd-8769-fa762a0469e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:23.594811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aa708-fcf7-42fd-8769-fa762a0469e3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:23.595097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aa708-fcf7-42fd-8769-fa762a0469e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:23.595375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aa708-fcf7-42fd-8769-fa762a0469e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:23.606119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307aa708-fcf7-42fd-8769-fa762a0469e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-307aa708-fcf7-42fd-8769-fa762a0469e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:20:23.611100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:23.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:23.613771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 619/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:20:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:23.619533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f7895-9bc5-4be8-b799-3012d90bb2fc None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:23.731171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f7895-9bc5-4be8-b799-3012d90bb2fc None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:23.760056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:23.776793 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:23.777422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:23.777713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:23.777982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:23.778647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] RBAC: Adding query filter params (name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:23.779035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:23.780474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:23.790651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 604/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:20:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:23.796766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aa708-fcf7-42fd-8769-fa762a0469e3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:23.843443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:23.923362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aa708-fcf7-42fd-8769-fa762a0469e3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:23.977704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:23.978789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:23.979297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:23.979830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:23.998783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:24.001297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:24.018970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:24.020155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:24.021077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:24.021077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:24.021077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:24.021077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 620/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:20:24.026947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:24.141066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e580943-c18d-4e04-a2c1-3b0e4ef8c81d None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:24.179772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:24.180955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:24.181243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:24.181571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:24.182219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:24.182545 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:24.185379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:24.198751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 605/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:24.205200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:24.317899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:24.325763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b5d6d-a0fd-4438-a47b-7359ceb82ce3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:24.368812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:24.369491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:24.369772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:24.370046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:24.379189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:24.380752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:20:24.397019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:24.398951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:24.400301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 621/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:24.405039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:24.459107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:24.523843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319a80c-01c4-406c-a26b-5f76c9f75ae8 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:24.563308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:24.563856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:24.564171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:24.564450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:24.565262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:24.565573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:24.567055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:24.588471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 606/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Oct 3 13:20:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:24.594799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:24.704389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628efd0a-be8b-40ce-be65-be68b81b80ba None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:24.742715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:24.743274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:24.743561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:24.743846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:24.755423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:24.757467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:24.771924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:24.773067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:24.774202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:24.774202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:24.774202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:24.774202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 622/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:24.779029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:24.901856 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7631325-501c-498d-9a04-6ea88cea944b None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:24.905341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:24.946568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:24.947418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:24.947712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:24.948243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:24.949332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:24.949790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:24.951759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:24.964222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 607/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:24.970291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:24.980818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:25.093475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129941c5-7b6f-4218-8d73-c7004d8e267d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:25.133018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:25.133704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:25.133982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:25.134256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:25.142034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:25.143513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:20:25.155415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:25.156751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:25.158585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 623/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:20:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:25.164813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:25.276212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0727c-8581-493f-b30f-014fa3445f96 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:25.316018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e0ab8-e892-47d5-af75-0fa26ce13878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:25.316640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e0ab8-e892-47d5-af75-0fa26ce13878 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:25.316924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e0ab8-e892-47d5-af75-0fa26ce13878 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:25.317199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e0ab8-e892-47d5-af75-0fa26ce13878 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:25.317948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e0ab8-e892-47d5-af75-0fa26ce13878 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:25.318306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e0ab8-e892-47d5-af75-0fa26ce13878 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:25.319864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e0ab8-e892-47d5-af75-0fa26ce13878 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:25.333207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 608/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:20:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:25.339397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:25.416801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:25.456703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64ce13-7bbc-4f42-abbf-f8e10d142977 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:25.493658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d951af0-4561-462c-8052-b57f44c2015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:25.494336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951af0-4561-462c-8052-b57f44c2015d None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:25.494622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951af0-4561-462c-8052-b57f44c2015d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:25.494905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d951af0-4561-462c-8052-b57f44c2015d None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/7bca29b8758b408da2bf05face26b551/roles/c49c853797c4458d84ca51c515a08755` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:25.511931 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d951af0-4561-462c-8052-b57f44c2015d None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=7bca29b8758b408da2bf05face26b551, role_id=c49c853797c4458d84ca51c515a08755)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:25.513690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d951af0-4561-462c-8052-b57f44c2015d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:25.530242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 624/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:20:25 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/7bca29b8758b408da2bf05face26b551/roles/c49c853797c4458d84ca51c515a08755 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:20:27.812881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fa28e-dd43-4152-93f6-5edc1f9e02cd None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:27.813542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fa28e-dd43-4152-93f6-5edc1f9e02cd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:27.813542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fa28e-dd43-4152-93f6-5edc1f9e02cd None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:27.814914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 609/1233] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:27.821533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36829b-73db-4e71-9c67-e425554e9ca1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:27.822167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36829b-73db-4e71-9c67-e425554e9ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:27.822167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36829b-73db-4e71-9c67-e425554e9ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:27.842109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4b36829b-73db-4e71-9c67-e425554e9ca1 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:27.843370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4b36829b-73db-4e71-9c67-e425554e9ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:27.928105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 625/1234] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:20:28.221316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fa28e-dd43-4152-93f6-5edc1f9e02cd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:28.250761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:28.341962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fa28e-dd43-4152-93f6-5edc1f9e02cd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:28.380935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828bea3-7727-4abe-847c-1db784ba1137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:28.381686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828bea3-7727-4abe-847c-1db784ba1137 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:28.382086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828bea3-7727-4abe-847c-1db784ba1137 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:28.382358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828bea3-7727-4abe-847c-1db784ba1137 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:28.395442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828bea3-7727-4abe-847c-1db784ba1137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-1828bea3-7727-4abe-847c-1db784ba1137 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:20:28.401348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:28.402502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:28.403358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 610/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Mon Oct 3 13:20:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:28.409201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36829b-73db-4e71-9c67-e425554e9ca1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:28.462181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:28.523212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36829b-73db-4e71-9c67-e425554e9ca1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:28.568187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:28.569723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:28.570345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:28.570956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:28.572037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:28.572436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:28.574596 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:28.586715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 626/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Mon Oct 3 13:20:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:28.593320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828bea3-7727-4abe-847c-1db784ba1137 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:28.695115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:28.714906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828bea3-7727-4abe-847c-1db784ba1137 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:28.755344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:28.755992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:28.756356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:28.757334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:28.769148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:28.770731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:28.786554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:28.787657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:28.789243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:28.789243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:28.789243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:28.789243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 611/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:28.795983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:28.916421 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1863f9-2997-4fdb-be88-f067b1c7d052 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:28.927250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:28.958008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:28.958628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:28.958953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:28.959332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:28.960061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:28.960681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:28.962485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:28.985821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 627/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:28.987254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:29.108552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5935b5c-2951-4b73-ad12-0a45b491621f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:29.149120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:29.149798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:29.150078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:29.150345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:29.153901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:29.160283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:29.162090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:20:29.175335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:29.176629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:29.177673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 612/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:29.186302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:29.303465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18633ef-0932-496a-8dd4-50e3f1f692e3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:29.343663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:29.344308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:29.344588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:29.344862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:29.345666 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:29.345979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:29.347513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:29.369284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 628/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Oct 3 13:20:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:29.375781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:29.497333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a89d61-56eb-48e0-baaa-ff6a3bea2a58 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:29.538358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6985afe-1a5f-433d-b80a-d0ae08d4b42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:29.539188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6985afe-1a5f-433d-b80a-d0ae08d4b42c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:29.539509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6985afe-1a5f-433d-b80a-d0ae08d4b42c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:29.539785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6985afe-1a5f-433d-b80a-d0ae08d4b42c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:29.609283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:29.617442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6985afe-1a5f-433d-b80a-d0ae08d4b42c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:29.618945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6985afe-1a5f-433d-b80a-d0ae08d4b42c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:29.735462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 613/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:20:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 13:20:29.743287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:29.775212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:29.858671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3642f70-ab60-4e11-9d86-c02d07abe98f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:29.898558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:29.899283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:29.899624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:29.899908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:29.917624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:29.920895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:29.938458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:29.939843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:29.941057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:29.941057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:29.941057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:29.941057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 629/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:29.946816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6985afe-1a5f-433d-b80a-d0ae08d4b42c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:30.052956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:30.060106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6985afe-1a5f-433d-b80a-d0ae08d4b42c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:30.097018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:30.097670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:30.097952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:30.098226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:30.098869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:30.099178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:30.101204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:30.112900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 614/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:30.119603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:30.210824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:30.238681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1eaf3-715b-4d63-b032-9a5f0f414997 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:30.278579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:30.279254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:30.279788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:30.280082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:30.288041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:30.291060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:20:30.309936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:30.311113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:30.312207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:20:30.312207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:20:30.312207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:20:30.312207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:20:30.312207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:20:30.312207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:20:30.312207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:30.312207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:30.312660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 630/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:20:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:30.318556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:30.454471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6d839-f30f-4cd5-bd5b-25d383deb303 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:30.492358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c3056-a4e0-4a8e-8f03-4b61058e3d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:30.492981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c3056-a4e0-4a8e-8f03-4b61058e3d1f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:30.493267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c3056-a4e0-4a8e-8f03-4b61058e3d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:30.493577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c3056-a4e0-4a8e-8f03-4b61058e3d1f None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:30.494272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c3056-a4e0-4a8e-8f03-4b61058e3d1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:30.494611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c3056-a4e0-4a8e-8f03-4b61058e3d1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:30.496148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c3056-a4e0-4a8e-8f03-4b61058e3d1f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:30.510858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 615/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:20:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:20:30.517274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:30.642854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de34d98-c8a7-470a-b502-f7e08cc777d5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:30.682940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438bb76-c3b2-4cc4-abd5-cbfbd4543888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:30.683550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bb76-c3b2-4cc4-abd5-cbfbd4543888 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:30.683838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bb76-c3b2-4cc4-abd5-cbfbd4543888 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:30.684148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438bb76-c3b2-4cc4-abd5-cbfbd4543888 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:30.684852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438bb76-c3b2-4cc4-abd5-cbfbd4543888 None admin] RBAC: Adding query filter params (role.id=c49c853797c4458d84ca51c515a08755, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=7bca29b8758b408da2bf05face26b551) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:30.685165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438bb76-c3b2-4cc4-abd5-cbfbd4543888 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:30.686670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438bb76-c3b2-4cc4-abd5-cbfbd4543888 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:30.693018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 631/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:20:30 2022] GET /identity/v3/role_assignments?user.id=7bca29b8758b408da2bf05face26b551&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=c49c853797c4458d84ca51c515a08755 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:20:33.196288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef6c1a-514d-4985-82fb-1bfde16f3088 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:33.196821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef6c1a-514d-4985-82fb-1bfde16f3088 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:33.196821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef6c1a-514d-4985-82fb-1bfde16f3088 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:33.198229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 616/1247] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:20:33.206893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c593b-5869-43d9-9e5b-507c1239ba79 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:33.207410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c593b-5869-43d9-9e5b-507c1239ba79 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:33.207669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c593b-5869-43d9-9e5b-507c1239ba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:33.214339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:33.228859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-805c593b-5869-43d9-9e5b-507c1239ba79 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:33.230116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-805c593b-5869-43d9-9e5b-507c1239ba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:33.302826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 632/1248] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:20:33.583260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef6c1a-514d-4985-82fb-1bfde16f3088 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:33.602377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:33.698793 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef6c1a-514d-4985-82fb-1bfde16f3088 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:33.734674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d12006-2e3f-44fe-b573-23a63bb76ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:33.735312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d12006-2e3f-44fe-b573-23a63bb76ebf None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:33.735588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d12006-2e3f-44fe-b573-23a63bb76ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:33.735855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d12006-2e3f-44fe-b573-23a63bb76ebf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:33.747716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d12006-2e3f-44fe-b573-23a63bb76ebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-54d12006-2e3f-44fe-b573-23a63bb76ebf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:20:33.752453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:33.753903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:33.758497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 617/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:20:33.760048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c593b-5869-43d9-9e5b-507c1239ba79 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:33.875930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c593b-5869-43d9-9e5b-507c1239ba79 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:33.914804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:33.923632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:33.924567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:33.924916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:33.925305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:33.926186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:33.926640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:33.928564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:33.939701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 633/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:33.946808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d12006-2e3f-44fe-b573-23a63bb76ebf None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:34.040371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:34.066903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d12006-2e3f-44fe-b573-23a63bb76ebf None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:34.108999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:34.109875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:34.110202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:34.110555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:34.122858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:34.124341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:34.141489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:34.142709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:34.144075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:34.144075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:34.144075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:34.144075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 618/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:20:34.149420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:34.256087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d716739-aac4-4e1f-8f99-cbf084e3923e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:34.293670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:34.294291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:34.294575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:34.294849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:34.295504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:34.295874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:34.297210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:34.310867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 634/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:34.316865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:34.428307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485c24c-7068-4784-a62f-169855204f8c None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:34.456608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:34.465312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:34.466001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:34.466286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:34.466564 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:34.474943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:34.477010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:20:34.489231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:34.490283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:34.492658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 619/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:34.496952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:34.562389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:34.614537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9190f-da89-47b8-8bd2-6ce6439ec701 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:34.644477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:34.645092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:34.645373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:34.645679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:34.646413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:34.646718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:34.649013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:34.668815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 635/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Oct 3 13:20:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:34.674841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:34.779020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b4fa3-9935-4dec-9f8a-4260ea8f51f1 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:34.814921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e4ba7-2005-43e6-8751-022e03259b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:34.815551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e4ba7-2005-43e6-8751-022e03259b51 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:34.815835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e4ba7-2005-43e6-8751-022e03259b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:34.816120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e4ba7-2005-43e6-8751-022e03259b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:34.871808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:34.885013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e4ba7-2005-43e6-8751-022e03259b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:34.886539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e4ba7-2005-43e6-8751-022e03259b51 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:34.992955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 620/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:20:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 03 13:20:34.998993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:35.103385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ed611-2335-4ccf-bfb1-be49cf54891d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:35.116195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:35.143920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:35.144555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:35.144841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:35.145120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:35.155798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:35.157265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:35.171511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:35.176279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:35.177272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:35.177272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:35.177272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:35.177272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 636/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:35.178592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e4ba7-2005-43e6-8751-022e03259b51 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:35.265715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:35.277782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e4ba7-2005-43e6-8751-022e03259b51 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:35.311819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:35.312431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:35.312715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:35.312990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:35.313656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:35.313963 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:35.315341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:35.327746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 621/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:20:35.333494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:35.433952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27911ba-2e25-455e-9d39-6e3d86a4cb07 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:35.471218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:35.471887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:35.472168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:35.472441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:35.480339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:35.481939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:20:35.494639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:35.495786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:35.496910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 637/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:20:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:35.502731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:35.611904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281967a-9975-4a3f-bb67-9521b4e731ca None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:35.647209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348c831-8a11-4994-a24f-2d0008f0c809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:35.647824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348c831-8a11-4994-a24f-2d0008f0c809 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:35.648103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348c831-8a11-4994-a24f-2d0008f0c809 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:35.648379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348c831-8a11-4994-a24f-2d0008f0c809 None admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:35.649080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348c831-8a11-4994-a24f-2d0008f0c809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:35.649389 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348c831-8a11-4994-a24f-2d0008f0c809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:35.650862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348c831-8a11-4994-a24f-2d0008f0c809 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:35.663687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 622/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:20:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:35.669970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:35.701557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:35.776987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007087c4-0e4d-43f4-ba76-8aace9153afe None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:35.811439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc11d9c-3a0e-4d41-b1d3-9e13a1d8debb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:35.812052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11d9c-3a0e-4d41-b1d3-9e13a1d8debb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:35.812337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11d9c-3a0e-4d41-b1d3-9e13a1d8debb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:35.812613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc11d9c-3a0e-4d41-b1d3-9e13a1d8debb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:35.813372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc11d9c-3a0e-4d41-b1d3-9e13a1d8debb None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=7bca29b8758b408da2bf05face26b551) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:35.813715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc11d9c-3a0e-4d41-b1d3-9e13a1d8debb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:35.815157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc11d9c-3a0e-4d41-b1d3-9e13a1d8debb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:35.820633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 638/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:20:35 2022] GET /identity/v3/role_assignments?user.id=7bca29b8758b408da2bf05face26b551&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:38.002994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd0ed-ff8d-493f-8b7d-0c3657859b46 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:38.003820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd0ed-ff8d-493f-8b7d-0c3657859b46 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:38.003820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fd0ed-ff8d-493f-8b7d-0c3657859b46 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:38.005148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 623/1261] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:38.010302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44984f98-639b-42f2-803c-238cad4a6ba6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:38.010719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44984f98-639b-42f2-803c-238cad4a6ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:38.010866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44984f98-639b-42f2-803c-238cad4a6ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:38.029203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-44984f98-639b-42f2-803c-238cad4a6ba6 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:38.030495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-44984f98-639b-42f2-803c-238cad4a6ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:38.103396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 639/1262] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:20:38.377130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fd0ed-ff8d-493f-8b7d-0c3657859b46 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:38.388319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:38.481249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2fd0ed-ff8d-493f-8b7d-0c3657859b46 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:38.515642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db16051-8875-4280-8a7a-cf7d77ace5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:38.516280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db16051-8875-4280-8a7a-cf7d77ace5cd None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:38.516564 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db16051-8875-4280-8a7a-cf7d77ace5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:38.516840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db16051-8875-4280-8a7a-cf7d77ace5cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:38.527416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db16051-8875-4280-8a7a-cf7d77ace5cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-5db16051-8875-4280-8a7a-cf7d77ace5cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:20:38.532220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:38.533443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:38.534358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 624/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:20:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:38.540470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44984f98-639b-42f2-803c-238cad4a6ba6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:38.558472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:38.644654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44984f98-639b-42f2-803c-238cad4a6ba6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:38.680817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:38.681481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:38.681765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:38.682043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:38.682745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:38.683059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:38.684503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:38.694284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 640/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:20:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:38.700503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db16051-8875-4280-8a7a-cf7d77ace5cd None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:38.772434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:38.806246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db16051-8875-4280-8a7a-cf7d77ace5cd None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:38.840824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:38.841483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:38.841765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:38.842042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:38.851953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:38.853404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:38.867582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:38.868764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:38.869899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:38.869899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:38.869899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:38.869899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 625/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:38.874588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:38.961056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:38.980208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67324c07-a6f9-41c8-9094-420b29c8ef92 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:39.014464 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:39.015137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:39.015338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:39.015617 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:39.016259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:39.016567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:39.017970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:39.029742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 641/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:20:39.035582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:39.134953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6a353-f7f0-41f6-bb5f-6e753f3b8be4 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:39.154088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:39.167918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:39.168545 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:39.168825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:39.169096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:39.176426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:39.177894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:20:39.189268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:39.190422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:39.191430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 626/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:39.196749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:39.305786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1390e0-a01f-455c-b499-06b6f087c87c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:39.341188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:39.341842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:39.342132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:39.342410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:39.343149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:39.343466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:39.344923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:39.354389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:39.362911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 642/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Oct 3 13:20:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:39.369295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:39.467443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb357fb-9da9-4a06-b663-5fcac76931b5 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:39.500840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:39.501493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:39.501777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:39.502050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:39.513391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:39.514994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:39.530407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:39.531607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:39.532580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:39.532580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:39.532580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:39.532580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 627/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:39.539142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:39.637672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e99b9-0dae-4457-9fb5-24509c7608d7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:39.671367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:39.671967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:39.672248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:39.672522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:39.673155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:39.673482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:39.674909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:39.686743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 643/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:39.692552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:39.728076 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:39.797291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee164f-af16-41e3-8451-717533b672f0 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:39.829349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:39.830005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:39.830287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:39.830561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:39.837714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:39.839169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:20:39.850700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:39.851846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:39.852978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 628/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:39.857797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:39.917381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:39.960401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cc594-4d49-455b-8fc0-78a73a5ef7cb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:39.993076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431630ce-3c74-46b2-af78-ec1c12f77835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:39.993704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431630ce-3c74-46b2-af78-ec1c12f77835 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:39.993986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431630ce-3c74-46b2-af78-ec1c12f77835 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:39.994261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431630ce-3c74-46b2-af78-ec1c12f77835 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:39.994944 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431630ce-3c74-46b2-af78-ec1c12f77835 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:39.995250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431630ce-3c74-46b2-af78-ec1c12f77835 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:39.996679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431630ce-3c74-46b2-af78-ec1c12f77835 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:40.008584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 644/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:40.014701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:40.108765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:40.121704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e46b8a-e382-499d-af3d-40c09d58e39f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:40.156488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9a75c-d37a-471c-a9d7-591ca805eb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:40.157147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9a75c-d37a-471c-a9d7-591ca805eb23 None admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:40.157438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9a75c-d37a-471c-a9d7-591ca805eb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:40.157743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9a75c-d37a-471c-a9d7-591ca805eb23 None admin] PATH_INFO: `/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/7bca29b8758b408da2bf05face26b551/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:40.159640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9a75c-d37a-471c-a9d7-591ca805eb23 None admin] RBAC: Authorizing `identity:create_grant(project_id=272f7bf825ad43a7a82311d1db6ae41e, user_id=7bca29b8758b408da2bf05face26b551, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:40.161071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c9a75c-d37a-471c-a9d7-591ca805eb23 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:40.173902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 629/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Mon Oct 3 13:20:40 2022] PUT /identity/v3/projects/272f7bf825ad43a7a82311d1db6ae41e/users/7bca29b8758b408da2bf05face26b551/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:20:42.359153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e853-a33d-4530-a2ba-6a81775662f7 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:42.359701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e853-a33d-4530-a2ba-6a81775662f7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:42.359701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e853-a33d-4530-a2ba-6a81775662f7 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:42.361068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 645/1274] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:42.367065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a74b8-d2d2-4347-bc5b-50b4729df223 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:42.367706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a74b8-d2d2-4347-bc5b-50b4729df223 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:42.367706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a74b8-d2d2-4347-bc5b-50b4729df223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:42.387138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1c1a74b8-d2d2-4347-bc5b-50b4729df223 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:42.388350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1c1a74b8-d2d2-4347-bc5b-50b4729df223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:42.459017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 630/1275] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:20:42.733734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e853-a33d-4530-a2ba-6a81775662f7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:42.843613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e853-a33d-4530-a2ba-6a81775662f7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:42.862817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:42.883650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01933dcd-0dd2-44e9-9559-c0eee6cc311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:42.884412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01933dcd-0dd2-44e9-9559-c0eee6cc311f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:42.884749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01933dcd-0dd2-44e9-9559-c0eee6cc311f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:42.885148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01933dcd-0dd2-44e9-9559-c0eee6cc311f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:42.896866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01933dcd-0dd2-44e9-9559-c0eee6cc311f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-01933dcd-0dd2-44e9-9559-c0eee6cc311f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 03 13:20:42.901903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:42.903041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:42.903981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 646/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Mon Oct 3 13:20:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 03 13:20:42.909542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a74b8-d2d2-4347-bc5b-50b4729df223 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:43.010562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:43.030126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a74b8-d2d2-4347-bc5b-50b4729df223 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:43.064741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:43.065369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:43.065687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:43.065962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:43.066630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:43.066936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:43.068321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:43.077001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 631/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Mon Oct 3 13:20:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:43.083135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01933dcd-0dd2-44e9-9559-c0eee6cc311f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:43.184736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01933dcd-0dd2-44e9-9559-c0eee6cc311f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:43.219593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:43.220235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:43.220517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:43.220792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:43.230884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:43.232352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:43.247584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:43.249247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:43.250538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:43.250538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:43.250538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:43.250538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 647/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:43.255154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:43.360730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66072cd-1dc6-4111-821d-594d1ce38b85 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:43.394965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:43.395620 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:43.395854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:43.396129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:43.396955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:43.397259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:43.398848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:43.410783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 632/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:43.417354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:43.448287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:43.520041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2db2da-a65b-4623-a1d6-8b0f56d86eaf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:43.555098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:43.555754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:43.556069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:43.556373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:43.564933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:43.566456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 03 13:20:43.579987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:43.581198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:43.582253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 648/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:43.587415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:43.591364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:43.691208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302c213-3809-4af4-b503-7086e4187671 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:43.730093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:43.730711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:43.730997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:43.731276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:43.732015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:43.732327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:43.733790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:43.751553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 633/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Mon Oct 3 13:20:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:43.758092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:43.855904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:43.866968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e409cf-b6b1-42e2-8276-d33e4e36b117 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:43.902953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289fd1f-a529-43eb-9cb0-b48dafa442f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:43.903579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289fd1f-a529-43eb-9cb0-b48dafa442f7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:43.903869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289fd1f-a529-43eb-9cb0-b48dafa442f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:43.904149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289fd1f-a529-43eb-9cb0-b48dafa442f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:43.980205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289fd1f-a529-43eb-9cb0-b48dafa442f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:43.981710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289fd1f-a529-43eb-9cb0-b48dafa442f7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:44.072226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:44.082909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 649/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Mon Oct 3 13:20:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 03 13:20:44.089985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:44.156058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:44.197359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdcc20-8871-4605-9046-83a36319787f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:44.232615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:44.233258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:44.233573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:44.233848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:44.244940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:44.246438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 03 13:20:44.262260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:44.263328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 03 13:20:44.264252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 03 13:20:44.264252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 03 13:20:44.264252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:44.264252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 634/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:44.270512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289fd1f-a529-43eb-9cb0-b48dafa442f7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:44.374476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289fd1f-a529-43eb-9cb0-b48dafa442f7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:44.412786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:44.413300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:44.413618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:44.413900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:44.414558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:44.414868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:44.416249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:44.431664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 650/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Mon Oct 3 13:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:44.439234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:44.552769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dbdb3-db63-45cf-83d3-65f32f6ccf9e None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:44.567165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:44.593221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:44.593927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:44.594212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:44.594488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:44.603522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:44.605015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 03 13:20:44.617780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:44.618936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: ERROR keystone.server.flask.application Oct 03 13:20:44.620264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 635/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Mon Oct 3 13:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:44.626200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:44.684716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:44.742185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a723c1-ad8d-4890-95f8-231bd1dfae64 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:44.778220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb40e3e-fdc8-4358-b580-82353c40dfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:44.778831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb40e3e-fdc8-4358-b580-82353c40dfcc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:44.779118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb40e3e-fdc8-4358-b580-82353c40dfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:44.779400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb40e3e-fdc8-4358-b580-82353c40dfcc None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:44.780210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb40e3e-fdc8-4358-b580-82353c40dfcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:44.780525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb40e3e-fdc8-4358-b580-82353c40dfcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:44.782034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb40e3e-fdc8-4358-b580-82353c40dfcc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:44.794427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 651/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Mon Oct 3 13:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:44.800741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:44.905674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652fa23-9df9-4e11-a3ae-451776f1c133 None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:44.944593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1802311-8bae-4931-a377-ab8a59c27b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:44.945057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1802311-8bae-4931-a377-ab8a59c27b63 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:44.945337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1802311-8bae-4931-a377-ab8a59c27b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:44.945645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1802311-8bae-4931-a377-ab8a59c27b63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:44.946379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1802311-8bae-4931-a377-ab8a59c27b63 None admin] RBAC: Adding query filter params (role.id=cdb47dc2f693403f958bbca4dab63130, scope.project.id=272f7bf825ad43a7a82311d1db6ae41e, user.id=7bca29b8758b408da2bf05face26b551) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:44.946690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1802311-8bae-4931-a377-ab8a59c27b63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:44.948142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1802311-8bae-4931-a377-ab8a59c27b63 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:44.954325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 636/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Mon Oct 3 13:20:44 2022] GET /identity/v3/role_assignments?user.id=7bca29b8758b408da2bf05face26b551&scope.project.id=272f7bf825ad43a7a82311d1db6ae41e&role.id=cdb47dc2f693403f958bbca4dab63130 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:47.214068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd4331-c8e0-4c7a-8b59-6552919388e7 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:47.214654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd4331-c8e0-4c7a-8b59-6552919388e7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:47.214654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd4331-c8e0-4c7a-8b59-6552919388e7 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:47.215975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 652/1288] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:47.222622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33217a7b-d5e1-4be4-bd8f-100db0beb727 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:47.223117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33217a7b-d5e1-4be4-bd8f-100db0beb727 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:47.223502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33217a7b-d5e1-4be4-bd8f-100db0beb727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:47.244300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-33217a7b-d5e1-4be4-bd8f-100db0beb727 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:47.245579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-33217a7b-d5e1-4be4-bd8f-100db0beb727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:47.277991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:47.321769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 637/1289] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:20:47.625919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd4331-c8e0-4c7a-8b59-6552919388e7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:47.743832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd4331-c8e0-4c7a-8b59-6552919388e7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:47.756214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:47.790184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:47.791048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:47.791338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:47.791619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:47.792303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:47.794081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:47.805402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:47.806837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:47.808156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 653/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:20:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:47.812859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33217a7b-d5e1-4be4-bd8f-100db0beb727 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:47.947130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33217a7b-d5e1-4be4-bd8f-100db0beb727 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:47.997897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388f670-0429-411a-b1d2-78dba8b3eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:47.998829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f670-0429-411a-b1d2-78dba8b3eb63 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:47.999309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f670-0429-411a-b1d2-78dba8b3eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:47.999784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388f670-0429-411a-b1d2-78dba8b3eb63 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:48.000742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388f670-0429-411a-b1d2-78dba8b3eb63 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:48.001323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388f670-0429-411a-b1d2-78dba8b3eb63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:48.003725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388f670-0429-411a-b1d2-78dba8b3eb63 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:48.012421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 638/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Mon Oct 3 13:20:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:48.020015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:48.148126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f02e74-31fe-4c86-ac8c-8cbcaa75aa90 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:48.199517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b52c7-7f9c-4a18-9c74-510fe25adb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:48.200373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b52c7-7f9c-4a18-9c74-510fe25adb7f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:48.200718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b52c7-7f9c-4a18-9c74-510fe25adb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:48.201114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b52c7-7f9c-4a18-9c74-510fe25adb7f None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:48.201927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b52c7-7f9c-4a18-9c74-510fe25adb7f None admin] RBAC: Adding query filter params (type=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:48.202370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b52c7-7f9c-4a18-9c74-510fe25adb7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:48.204819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b52c7-7f9c-4a18-9c74-510fe25adb7f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:48.212838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 654/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Mon Oct 3 13:20:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:50.623265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329da0b4-bb6c-4bb4-8509-5d7895f00734 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:50.623784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329da0b4-bb6c-4bb4-8509-5d7895f00734 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:50.623784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329da0b4-bb6c-4bb4-8509-5d7895f00734 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:50.625207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 639/1293] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:50.631794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2ac51-2003-4795-a58c-bad7396243bb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:50.632350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2ac51-2003-4795-a58c-bad7396243bb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:50.632649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2ac51-2003-4795-a58c-bad7396243bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:50.652808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-62e2ac51-2003-4795-a58c-bad7396243bb None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:50.654184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-62e2ac51-2003-4795-a58c-bad7396243bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:50.689946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:50.732452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 655/1294] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 03 13:20:51.035197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329da0b4-bb6c-4bb4-8509-5d7895f00734 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:51.072152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:51.157410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329da0b4-bb6c-4bb4-8509-5d7895f00734 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:51.194102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f98c3-9fb8-4492-87cb-8731849971c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:51.194536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f98c3-9fb8-4492-87cb-8731849971c3 None admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:51.194819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f98c3-9fb8-4492-87cb-8731849971c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:51.195138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f98c3-9fb8-4492-87cb-8731849971c3 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:51.195815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f98c3-9fb8-4492-87cb-8731849971c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:51.197305 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f98c3-9fb8-4492-87cb-8731849971c3 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:51.213111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 640/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Mon Oct 3 13:20:51 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:20:53.585130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dbf7a-3b8b-42a6-9adf-2e24cb2675fd None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:53.585776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dbf7a-3b8b-42a6-9adf-2e24cb2675fd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:53.585956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dbf7a-3b8b-42a6-9adf-2e24cb2675fd None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:53.587361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 656/1296] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:53.593771 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4b3a-7997-4cdf-8484-34ed81188ab9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:53.594227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4b3a-7997-4cdf-8484-34ed81188ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:53.594336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4b3a-7997-4cdf-8484-34ed81188ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:53.614942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d57d4b3a-7997-4cdf-8484-34ed81188ab9 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:53.616283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d57d4b3a-7997-4cdf-8484-34ed81188ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:53.702037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 641/1297] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:20:54.000591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dbf7a-3b8b-42a6-9adf-2e24cb2675fd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:54.136276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dbf7a-3b8b-42a6-9adf-2e24cb2675fd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:54.175698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:54.176342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:54.176628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:54.176977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:54.178200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:54.179943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:54.190064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:54.191372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:54.192378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 657/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:20:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:54.198498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d4b3a-7997-4cdf-8484-34ed81188ab9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:54.220456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:54.322268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d4b3a-7997-4cdf-8484-34ed81188ab9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:54.361821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:54.362606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:54.362897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:54.363285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:54.364110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:54.364594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:54.367752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:54.377888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 642/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Mon Oct 3 13:20:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:54.384835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:54.424449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:54.505634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e126485-fd4d-4553-950c-43e92005ce5f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:54.543055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1477cf9-8b5b-4402-8c3a-462ba94b3285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:54.543689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477cf9-8b5b-4402-8c3a-462ba94b3285 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:54.544046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477cf9-8b5b-4402-8c3a-462ba94b3285 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:54.544328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477cf9-8b5b-4402-8c3a-462ba94b3285 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:54.545107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1477cf9-8b5b-4402-8c3a-462ba94b3285 None admin] RBAC: Adding query filter params (interface=public, service_id=4353d969f08a4ec18872219a5de8535d, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:54.545590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1477cf9-8b5b-4402-8c3a-462ba94b3285 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:54.547057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1477cf9-8b5b-4402-8c3a-462ba94b3285 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:54.554314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 658/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Mon Oct 3 13:20:54 2022] GET /identity/v3/endpoints?service_id=4353d969f08a4ec18872219a5de8535d&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:54.560869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:54.673578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:54.703073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3ce3f-26c3-407c-9d84-0d357e3c186f None admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:54.748908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a8cca-2cfa-42ac-a87f-4a15d0830611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:54.749636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a8cca-2cfa-42ac-a87f-4a15d0830611 None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:54.749914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a8cca-2cfa-42ac-a87f-4a15d0830611 None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:54.750183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a8cca-2cfa-42ac-a87f-4a15d0830611 None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:54.750811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a8cca-2cfa-42ac-a87f-4a15d0830611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:54.752232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a8cca-2cfa-42ac-a87f-4a15d0830611 None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:54.763545 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 643/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Mon Oct 3 13:20:54 2022] GET /identity/v3/services => generated 2191 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:20:57.064009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726489b6-21f5-423b-96ed-cf1cfd4c4d65 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:57.064547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726489b6-21f5-423b-96ed-cf1cfd4c4d65 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:57.064547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726489b6-21f5-423b-96ed-cf1cfd4c4d65 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:57.065972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 659/1302] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:20:57.071630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a368a-8db2-491c-9bde-9ee85528ed8b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:57.072123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a368a-8db2-491c-9bde-9ee85528ed8b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:57.072216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a368a-8db2-491c-9bde-9ee85528ed8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:57.091768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1e5a368a-8db2-491c-9bde-9ee85528ed8b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:20:57.093003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1e5a368a-8db2-491c-9bde-9ee85528ed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:20:57.159653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 644/1303] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:20:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 03 13:20:57.432251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726489b6-21f5-423b-96ed-cf1cfd4c4d65 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:57.531926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:57.542718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726489b6-21f5-423b-96ed-cf1cfd4c4d65 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:57.576822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:57.577487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:57.577769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:57.578046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:57.578631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:57.580026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 03 13:20:57.588799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 03 13:20:57.590034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 03 13:20:57.591186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 660/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Mon Oct 3 13:20:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 03 13:20:57.595938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a368a-8db2-491c-9bde-9ee85528ed8b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:57.672765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:20:57.697081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a368a-8db2-491c-9bde-9ee85528ed8b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:57.735711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7936d-1604-4b33-8d11-c71c48f7760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:57.736324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7936d-1604-4b33-8d11-c71c48f7760f None admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:57.736608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7936d-1604-4b33-8d11-c71c48f7760f None admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:57.736884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7936d-1604-4b33-8d11-c71c48f7760f None admin] PATH_INFO: `/v3/services` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:57.737542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7936d-1604-4b33-8d11-c71c48f7760f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:20:57.737851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7936d-1604-4b33-8d11-c71c48f7760f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:57.739225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7936d-1604-4b33-8d11-c71c48f7760f None admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:57.744937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 645/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Mon Oct 3 13:20:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:20:57.750754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:20:57.850036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98c86d-4642-4b7d-ad9e-379c292bc479 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:20:57.883444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58e1b0-67ce-43b4-adc8-366422844132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:20:57.884051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58e1b0-67ce-43b4-adc8-366422844132 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:20:57.884337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58e1b0-67ce-43b4-adc8-366422844132 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:20:57.884615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58e1b0-67ce-43b4-adc8-366422844132 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:20:57.885300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58e1b0-67ce-43b4-adc8-366422844132 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:20:57.886699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58e1b0-67ce-43b4-adc8-366422844132 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:20:57.907894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 661/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:20:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:21:07.742080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=30 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:21:07.903397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:05.792737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0704f-3c4c-4bd6-8632-0b925c2719a6 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:05.793199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0704f-3c4c-4bd6-8632-0b925c2719a6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:05.793199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0704f-3c4c-4bd6-8632-0b925c2719a6 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:05.800658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da0f3c-7a3c-4633-ae97-9c3b9fef228a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:05.801109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da0f3c-7a3c-4633-ae97-9c3b9fef228a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:05.801199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da0f3c-7a3c-4633-ae97-9c3b9fef228a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:05.805897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 646/1307] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:06.058041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d1da0f3c-7a3c-4633-ae97-9c3b9fef228a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:06.059877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d1da0f3c-7a3c-4633-ae97-9c3b9fef228a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:06.199719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 662/1308] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:06.500205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43799fac-b343-4a0b-896f-4dc806e5959a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:06.500654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43799fac-b343-4a0b-896f-4dc806e5959a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:06.501066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43799fac-b343-4a0b-896f-4dc806e5959a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:06.503510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 647/1309] 10.222.0.58 () {54 vars in 964 bytes} [Mon Oct 3 13:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:06.510041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91caeba-dc3e-40a9-b61c-0cb33d5fa916 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:06.510449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91caeba-dc3e-40a9-b61c-0cb33d5fa916 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:06.510646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91caeba-dc3e-40a9-b61c-0cb33d5fa916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:06.612605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e91caeba-dc3e-40a9-b61c-0cb33d5fa916 None None] MFA Rules not processed for user `cf865cb3c7fc432884e4e308c397c83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:06.613968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e91caeba-dc3e-40a9-b61c-0cb33d5fa916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:06.638564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:06.704666 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 663/1310] 10.222.0.58 () {58 vars in 1061 bytes} [Mon Oct 3 13:22:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:06.715205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43799fac-b343-4a0b-896f-4dc806e5959a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:06.717139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-43799fac-b343-4a0b-896f-4dc806e5959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:06.848468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43799fac-b343-4a0b-896f-4dc806e5959a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:06.941041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ec6bb-2dab-4a62-be18-25958aed46d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:06.941903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec6bb-2dab-4a62-be18-25958aed46d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:06.942315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec6bb-2dab-4a62-be18-25958aed46d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:06.942691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ec6bb-2dab-4a62-be18-25958aed46d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:06.944485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-581ec6bb-2dab-4a62-be18-25958aed46d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:07.037183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:07.149846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ec6bb-2dab-4a62-be18-25958aed46d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:07.152771 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ec6bb-2dab-4a62-be18-25958aed46d3 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:07.188628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 648/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:09.783432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7083f1f-8e26-4c2a-9800-3d003a63cfa8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:09.784171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7083f1f-8e26-4c2a-9800-3d003a63cfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:09.784171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7083f1f-8e26-4c2a-9800-3d003a63cfa8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:09.785560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 664/1312] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:09.791077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971e34-b3bc-40e8-bdc4-7ff9ecf66cfc None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:09.791569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971e34-b3bc-40e8-bdc4-7ff9ecf66cfc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:09.791651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0971e34-b3bc-40e8-bdc4-7ff9ecf66cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:09.867432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d0971e34-b3bc-40e8-bdc4-7ff9ecf66cfc None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:09.868674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d0971e34-b3bc-40e8-bdc4-7ff9ecf66cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:09.981310 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 649/1313] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:10.266160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7083f1f-8e26-4c2a-9800-3d003a63cfa8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:10.267839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d7083f1f-8e26-4c2a-9800-3d003a63cfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:10.388349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7083f1f-8e26-4c2a-9800-3d003a63cfa8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:10.466449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:10.469036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f561-b4c9-41aa-aab0-c8fde2697a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:10.469747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f561-b4c9-41aa-aab0-c8fde2697a00 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:10.470222 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f561-b4c9-41aa-aab0-c8fde2697a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:10.470691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f561-b4c9-41aa-aab0-c8fde2697a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:10.473390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3236f561-b4c9-41aa-aab0-c8fde2697a00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:10.600778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f561-b4c9-41aa-aab0-c8fde2697a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:10.602240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236f561-b4c9-41aa-aab0-c8fde2697a00 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:10.623564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 665/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:13.233695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1421d-ae3b-425b-bb11-b95d1f8cdbe5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:13.234349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1421d-ae3b-425b-bb11-b95d1f8cdbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:13.234349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1421d-ae3b-425b-bb11-b95d1f8cdbe5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:13.235652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 650/1315] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:13.242532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08766bc-4a75-4cef-899d-96ca47de4a8f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:13.243252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08766bc-4a75-4cef-899d-96ca47de4a8f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:13.243252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08766bc-4a75-4cef-899d-96ca47de4a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:13.323539 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e08766bc-4a75-4cef-899d-96ca47de4a8f None None] MFA Rules not processed for user `02a67abf7e2341ab9df44025a4aa90ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:13.324806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e08766bc-4a75-4cef-899d-96ca47de4a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:13.384872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:13.438021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 666/1316] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:13.732084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1421d-ae3b-425b-bb11-b95d1f8cdbe5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:13.733658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-aac1421d-ae3b-425b-bb11-b95d1f8cdbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:13.743407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:13.842542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac1421d-ae3b-425b-bb11-b95d1f8cdbe5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:13.917776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884c9e-60d9-498d-86ee-e10001de98d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:13.918394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884c9e-60d9-498d-86ee-e10001de98d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:13.918675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884c9e-60d9-498d-86ee-e10001de98d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:13.918950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884c9e-60d9-498d-86ee-e10001de98d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:13.920553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-49884c9e-60d9-498d-86ee-e10001de98d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:13.985349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:14.032306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884c9e-60d9-498d-86ee-e10001de98d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:14.034068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884c9e-60d9-498d-86ee-e10001de98d6 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:14.052279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 651/1317] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:16.800622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9155f2-aaf9-436f-b9f7-eebccb6bbcc4 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:16.801192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9155f2-aaf9-436f-b9f7-eebccb6bbcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:16.801192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9155f2-aaf9-436f-b9f7-eebccb6bbcc4 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:16.802579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 667/1318] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:16.808057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0cc3e-03e4-4f6a-bddf-6804f0767314 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:16.808446 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0cc3e-03e4-4f6a-bddf-6804f0767314 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:16.808593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0cc3e-03e4-4f6a-bddf-6804f0767314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:16.879373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3bf0cc3e-03e4-4f6a-bddf-6804f0767314 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:16.880616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3bf0cc3e-03e4-4f6a-bddf-6804f0767314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:16.978843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 652/1319] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:17.281246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d77e6-335d-41e6-a7b7-97509d5d5fe8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:17.281563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d77e6-335d-41e6-a7b7-97509d5d5fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:17.281851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d77e6-335d-41e6-a7b7-97509d5d5fe8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:17.283803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 668/1320] 10.222.0.58 () {54 vars in 964 bytes} [Mon Oct 3 13:22:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:17.289979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a8569-8027-40cf-be78-b03f83e305ff None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:17.290271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a8569-8027-40cf-be78-b03f83e305ff None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:17.290550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a8569-8027-40cf-be78-b03f83e305ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:17.313022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:17.370327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-172a8569-8027-40cf-be78-b03f83e305ff None None] MFA Rules not processed for user `cf865cb3c7fc432884e4e308c397c83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:17.371562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-172a8569-8027-40cf-be78-b03f83e305ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:17.447229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 653/1321] 10.222.0.58 () {58 vars in 1061 bytes} [Mon Oct 3 13:22:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:17.456764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d77e6-335d-41e6-a7b7-97509d5d5fe8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:17.458333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8c3d77e6-335d-41e6-a7b7-97509d5d5fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:17.561752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d77e6-335d-41e6-a7b7-97509d5d5fe8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:17.635522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff016a20-9a00-4bf4-9d94-8b66e5f18595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:17.636133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff016a20-9a00-4bf4-9d94-8b66e5f18595 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:17.636410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff016a20-9a00-4bf4-9d94-8b66e5f18595 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:17.636683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff016a20-9a00-4bf4-9d94-8b66e5f18595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:17.638318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ff016a20-9a00-4bf4-9d94-8b66e5f18595 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:17.645580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:17.752417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff016a20-9a00-4bf4-9d94-8b66e5f18595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:17.753805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff016a20-9a00-4bf4-9d94-8b66e5f18595 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:17.771737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 669/1322] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:21.257603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d60ca-c751-4ee9-81a9-71cac7cd267c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:21.258185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d60ca-c751-4ee9-81a9-71cac7cd267c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:21.258185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d60ca-c751-4ee9-81a9-71cac7cd267c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:21.259566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 654/1323] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:21.265732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0f0f-6b54-4385-becd-6365195c1761 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:21.266296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0f0f-6b54-4385-becd-6365195c1761 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:21.266296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0f0f-6b54-4385-becd-6365195c1761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:21.346090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a6fd0f0f-6b54-4385-becd-6365195c1761 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:21.347581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a6fd0f0f-6b54-4385-becd-6365195c1761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:21.414726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:21.461426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 670/1324] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:21.744796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d60ca-c751-4ee9-81a9-71cac7cd267c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:21.746473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-eb2d60ca-c751-4ee9-81a9-71cac7cd267c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:21.862923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d60ca-c751-4ee9-81a9-71cac7cd267c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:21.878006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:21.944415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c393f8-d6c4-4ebe-bb86-d5f8c2624208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:21.945037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c393f8-d6c4-4ebe-bb86-d5f8c2624208 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:21.945313 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c393f8-d6c4-4ebe-bb86-d5f8c2624208 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:21.945610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c393f8-d6c4-4ebe-bb86-d5f8c2624208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:21.947205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-69c393f8-d6c4-4ebe-bb86-d5f8c2624208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:22.059315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c393f8-d6c4-4ebe-bb86-d5f8c2624208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:22.060743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c393f8-d6c4-4ebe-bb86-d5f8c2624208 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:22.078548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 655/1325] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:24.335423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba335119-2a7d-456a-af9d-8f0b3c1d0a88 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:24.335957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba335119-2a7d-456a-af9d-8f0b3c1d0a88 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:24.335957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba335119-2a7d-456a-af9d-8f0b3c1d0a88 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:24.339525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 671/1326] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:24.343998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04ed4e-1ac7-468d-a3d8-98fd383b4aba None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:24.344538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04ed4e-1ac7-468d-a3d8-98fd383b4aba None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:24.344538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04ed4e-1ac7-468d-a3d8-98fd383b4aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:24.401594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:24.426380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2d04ed4e-1ac7-468d-a3d8-98fd383b4aba None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:24.427580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2d04ed4e-1ac7-468d-a3d8-98fd383b4aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:24.563490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 656/1327] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:24.852276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba335119-2a7d-456a-af9d-8f0b3c1d0a88 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:24.853924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ba335119-2a7d-456a-af9d-8f0b3c1d0a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:24.972305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba335119-2a7d-456a-af9d-8f0b3c1d0a88 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:25.052085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2a877-640b-4db6-aa4f-0097ebc1b691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:25.052721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2a877-640b-4db6-aa4f-0097ebc1b691 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:25.053008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2a877-640b-4db6-aa4f-0097ebc1b691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:25.053286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2a877-640b-4db6-aa4f-0097ebc1b691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:25.054984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-47d2a877-640b-4db6-aa4f-0097ebc1b691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:25.076508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:25.194811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2a877-640b-4db6-aa4f-0097ebc1b691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:25.196262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2a877-640b-4db6-aa4f-0097ebc1b691 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:25.216517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 672/1328] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:27.544607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236faba-847c-4946-8754-29b5c9a2c386 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:27.546415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236faba-847c-4946-8754-29b5c9a2c386 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:27.547239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236faba-847c-4946-8754-29b5c9a2c386 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:27.550110 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 657/1329] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:27.556361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb1dd6-8574-4ee8-b4df-3f195cd686e3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:27.556852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb1dd6-8574-4ee8-b4df-3f195cd686e3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:27.556988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb1dd6-8574-4ee8-b4df-3f195cd686e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:27.637929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-81cb1dd6-8574-4ee8-b4df-3f195cd686e3 None None] MFA Rules not processed for user `02a67abf7e2341ab9df44025a4aa90ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:27.639176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-81cb1dd6-8574-4ee8-b4df-3f195cd686e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:27.709392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:27.744480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 673/1330] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:28.043008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236faba-847c-4946-8754-29b5c9a2c386 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:28.044485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8236faba-847c-4946-8754-29b5c9a2c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:28.175435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236faba-847c-4946-8754-29b5c9a2c386 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:28.200593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:28.259993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f17dc-ef90-4d58-bd8e-ee4d948896e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:28.260612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f17dc-ef90-4d58-bd8e-ee4d948896e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:28.260889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f17dc-ef90-4d58-bd8e-ee4d948896e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:28.261161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f17dc-ef90-4d58-bd8e-ee4d948896e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:28.262813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-526f17dc-ef90-4d58-bd8e-ee4d948896e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:28.420976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f17dc-ef90-4d58-bd8e-ee4d948896e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:28.424086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f17dc-ef90-4d58-bd8e-ee4d948896e7 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:28.461988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 658/1331] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:30.997345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e0c05-2c25-468e-afda-031abea325a2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:30.997920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e0c05-2c25-468e-afda-031abea325a2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:30.997920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e0c05-2c25-468e-afda-031abea325a2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:30.999300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 674/1332] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:31.005836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d92134-b108-47b0-bb51-c2f550e9f177 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:31.006286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d92134-b108-47b0-bb51-c2f550e9f177 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:31.006377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d92134-b108-47b0-bb51-c2f550e9f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:31.071836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:31.074794 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-21d92134-b108-47b0-bb51-c2f550e9f177 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:31.076574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-21d92134-b108-47b0-bb51-c2f550e9f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:31.180224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 659/1333] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:31.475660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e0c05-2c25-468e-afda-031abea325a2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:31.477335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e68e0c05-2c25-468e-afda-031abea325a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:31.605430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e0c05-2c25-468e-afda-031abea325a2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:31.685198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e811f-a5d3-49e3-b593-ecf69d40f6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:31.685841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e811f-a5d3-49e3-b593-ecf69d40f6de service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:31.686126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e811f-a5d3-49e3-b593-ecf69d40f6de service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:31.686403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e811f-a5d3-49e3-b593-ecf69d40f6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:31.688055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-198e811f-a5d3-49e3-b593-ecf69d40f6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:31.719954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:31.813359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e811f-a5d3-49e3-b593-ecf69d40f6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:31.815565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e811f-a5d3-49e3-b593-ecf69d40f6de service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:31.835847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 675/1334] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:34.170493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca18a8-2df3-43be-84e1-bedd6233175e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:34.171042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca18a8-2df3-43be-84e1-bedd6233175e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:34.171042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aca18a8-2df3-43be-84e1-bedd6233175e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:34.172421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 660/1335] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:34.178507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ad0cd-bfd4-44a8-8fcf-c08788f865eb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:34.179018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ad0cd-bfd4-44a8-8fcf-c08788f865eb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:34.179145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ad0cd-bfd4-44a8-8fcf-c08788f865eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:34.254624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cd4ad0cd-bfd4-44a8-8fcf-c08788f865eb None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:34.255891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cd4ad0cd-bfd4-44a8-8fcf-c08788f865eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:34.347287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:34.371450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 676/1336] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:34.668781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca18a8-2df3-43be-84e1-bedd6233175e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:34.670372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8aca18a8-2df3-43be-84e1-bedd6233175e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:34.782143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aca18a8-2df3-43be-84e1-bedd6233175e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:34.818843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:34.868885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd51db6-7c79-4295-9e8c-46f4be454056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:34.869526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51db6-7c79-4295-9e8c-46f4be454056 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:34.869807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51db6-7c79-4295-9e8c-46f4be454056 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:34.870077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd51db6-7c79-4295-9e8c-46f4be454056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:34.871683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ffd51db6-7c79-4295-9e8c-46f4be454056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:34.987649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd51db6-7c79-4295-9e8c-46f4be454056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:34.989112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd51db6-7c79-4295-9e8c-46f4be454056 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:35.008447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 661/1337] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:37.731334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cce5c-9c7f-4021-9414-6b15effefb93 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:37.731860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cce5c-9c7f-4021-9414-6b15effefb93 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:37.731860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cce5c-9c7f-4021-9414-6b15effefb93 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:37.733210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 677/1338] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:37.739357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558bebd-212c-473b-af35-46bec42382da None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:37.739889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558bebd-212c-473b-af35-46bec42382da None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:37.739889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558bebd-212c-473b-af35-46bec42382da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:37.815687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b558bebd-212c-473b-af35-46bec42382da None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:37.816933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b558bebd-212c-473b-af35-46bec42382da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:37.825697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:37.925214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 662/1339] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:38.210088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cce5c-9c7f-4021-9414-6b15effefb93 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:38.211667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7a6cce5c-9c7f-4021-9414-6b15effefb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:38.322882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6cce5c-9c7f-4021-9414-6b15effefb93 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:38.405402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1b3b1-f1eb-49b1-aed6-973717f5d804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:38.406052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b3b1-f1eb-49b1-aed6-973717f5d804 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:38.406338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b3b1-f1eb-49b1-aed6-973717f5d804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:38.406614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b3b1-f1eb-49b1-aed6-973717f5d804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:38.408313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e5d1b3b1-f1eb-49b1-aed6-973717f5d804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:38.444738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:38.520975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1b3b1-f1eb-49b1-aed6-973717f5d804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:38.522375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1b3b1-f1eb-49b1-aed6-973717f5d804 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:38.541355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 678/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:43.637919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8dd1-cde3-4f95-8ea5-870d67dd0cae None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:43.638821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8dd1-cde3-4f95-8ea5-870d67dd0cae None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:43.638821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad8dd1-cde3-4f95-8ea5-870d67dd0cae None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:43.640106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 663/1341] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:43.647513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49adbec-2b18-4c8f-9374-82eb7d11b0aa None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:43.648108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49adbec-2b18-4c8f-9374-82eb7d11b0aa None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:43.648690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49adbec-2b18-4c8f-9374-82eb7d11b0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:43.730981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f49adbec-2b18-4c8f-9374-82eb7d11b0aa None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:43.732224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f49adbec-2b18-4c8f-9374-82eb7d11b0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:43.833156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:43.848506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 679/1342] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:44.154094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad8dd1-cde3-4f95-8ea5-870d67dd0cae None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:44.155790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5fad8dd1-cde3-4f95-8ea5-870d67dd0cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:44.291229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad8dd1-cde3-4f95-8ea5-870d67dd0cae None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:44.341571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:44.394115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a160f-6625-4857-a96e-edf21b8eeefc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:44.395197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a160f-6625-4857-a96e-edf21b8eeefc service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:44.395593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a160f-6625-4857-a96e-edf21b8eeefc service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:44.396060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a160f-6625-4857-a96e-edf21b8eeefc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:44.399337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-261a160f-6625-4857-a96e-edf21b8eeefc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:44.615222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a160f-6625-4857-a96e-edf21b8eeefc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:44.616764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a160f-6625-4857-a96e-edf21b8eeefc service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:44.638878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 664/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:22:47.803534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8153-dbbc-478c-bcd8-44ff4f2fee19 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:47.804259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8153-dbbc-478c-bcd8-44ff4f2fee19 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:47.804259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf8153-dbbc-478c-bcd8-44ff4f2fee19 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:47.805777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 680/1344] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:47.811762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb22d42-c2ab-47d1-b97a-b2fa48a7d0bd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:47.812309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb22d42-c2ab-47d1-b97a-b2fa48a7d0bd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:47.812448 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb22d42-c2ab-47d1-b97a-b2fa48a7d0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:47.895915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8fb22d42-c2ab-47d1-b97a-b2fa48a7d0bd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:47.897213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8fb22d42-c2ab-47d1-b97a-b2fa48a7d0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:47.920078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:48.041544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 665/1345] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:48.515578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8153-dbbc-478c-bcd8-44ff4f2fee19 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:48.517434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-31cf8153-dbbc-478c-bcd8-44ff4f2fee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:48.701273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf8153-dbbc-478c-bcd8-44ff4f2fee19 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:48.793899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5422d3-4227-46be-b103-96d9d2c702d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:48.794543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5422d3-4227-46be-b103-96d9d2c702d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:48.794829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5422d3-4227-46be-b103-96d9d2c702d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:48.795115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5422d3-4227-46be-b103-96d9d2c702d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:48.796978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bf5422d3-4227-46be-b103-96d9d2c702d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:48.854564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:48.935620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5422d3-4227-46be-b103-96d9d2c702d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:48.937131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5422d3-4227-46be-b103-96d9d2c702d4 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:48.958557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 681/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:22:53.702237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003291c2-dab3-4f7e-a929-4a519b1fe4ba None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:53.702721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003291c2-dab3-4f7e-a929-4a519b1fe4ba None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:53.703090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003291c2-dab3-4f7e-a929-4a519b1fe4ba None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:53.704938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 666/1347] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:22:53.711235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f2608-d37a-419e-98a0-4c6cdf50fd31 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:53.711995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f2608-d37a-419e-98a0-4c6cdf50fd31 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:53.712227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f2608-d37a-419e-98a0-4c6cdf50fd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:53.796376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-840f2608-d37a-419e-98a0-4c6cdf50fd31 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:22:53.799045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-840f2608-d37a-419e-98a0-4c6cdf50fd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:53.942980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:53.946049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 682/1348] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:22:54.269939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003291c2-dab3-4f7e-a929-4a519b1fe4ba None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:22:54.272660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-003291c2-dab3-4f7e-a929-4a519b1fe4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:54.460618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003291c2-dab3-4f7e-a929-4a519b1fe4ba None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:22:54.524165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:22:54.553787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76a9f7-9c27-4c74-922b-8c3099e3dc97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:22:54.554406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76a9f7-9c27-4c74-922b-8c3099e3dc97 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:22:54.554690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76a9f7-9c27-4c74-922b-8c3099e3dc97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:22:54.554968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76a9f7-9c27-4c74-922b-8c3099e3dc97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:22:54.556638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4c76a9f7-9c27-4c74-922b-8c3099e3dc97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:22:54.730623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76a9f7-9c27-4c74-922b-8c3099e3dc97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:22:54.732099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76a9f7-9c27-4c74-922b-8c3099e3dc97 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:22:54.751859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 667/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:00.164740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905effb9-1c40-4daa-845f-8d26bd1507e5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:00.165237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905effb9-1c40-4daa-845f-8d26bd1507e5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:00.165417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905effb9-1c40-4daa-845f-8d26bd1507e5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:00.166766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 683/1350] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:00.172386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f7bd-5e92-488a-9a8e-de57f54d3c2b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:00.173523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f7bd-5e92-488a-9a8e-de57f54d3c2b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:00.173609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745f7bd-5e92-488a-9a8e-de57f54d3c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:00.307919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a745f7bd-5e92-488a-9a8e-de57f54d3c2b None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:00.310181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a745f7bd-5e92-488a-9a8e-de57f54d3c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:00.351105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:00.538765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 668/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:23:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:23:05.636011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0733a1f-7de8-4d00-9f2f-bb5bbefcd547 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:05.636972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0733a1f-7de8-4d00-9f2f-bb5bbefcd547 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:05.637396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0733a1f-7de8-4d00-9f2f-bb5bbefcd547 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:05.639835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 684/1352] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:23:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:05.646600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abb615-8380-4443-9ce1-dbaaa7ede9f7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:05.647825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abb615-8380-4443-9ce1-dbaaa7ede9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:05.648321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abb615-8380-4443-9ce1-dbaaa7ede9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:05.768203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-40abb615-8380-4443-9ce1-dbaaa7ede9f7 None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:05.769445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-40abb615-8380-4443-9ce1-dbaaa7ede9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:05.876003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:05.923513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 669/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:23:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:07.277242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565b689-ae8a-4f22-a0c0-629af1b1e3c3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:07.277839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565b689-ae8a-4f22-a0c0-629af1b1e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:07.277839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565b689-ae8a-4f22-a0c0-629af1b1e3c3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:07.279336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 685/1354] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:07.285793 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987393e-d304-4f62-a629-eb0046fe3eb7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:07.286300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987393e-d304-4f62-a629-eb0046fe3eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:07.286449 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987393e-d304-4f62-a629-eb0046fe3eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:07.386181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8987393e-d304-4f62-a629-eb0046fe3eb7 None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:07.387432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8987393e-d304-4f62-a629-eb0046fe3eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:07.541116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 670/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:23:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:11.436470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb383e0-fc08-427c-9b36-2ca82cdd389c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:11.437284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb383e0-fc08-427c-9b36-2ca82cdd389c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:11.437284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb383e0-fc08-427c-9b36-2ca82cdd389c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:11.438779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 686/1356] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:11.447945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0958d4-71c1-4f51-9e31-db12dcf3ddb3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:11.448841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0958d4-71c1-4f51-9e31-db12dcf3ddb3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:11.450354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0958d4-71c1-4f51-9e31-db12dcf3ddb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:11.526866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:11.566894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cd0958d4-71c1-4f51-9e31-db12dcf3ddb3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:11.568127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cd0958d4-71c1-4f51-9e31-db12dcf3ddb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:11.691682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 671/1357] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:11.997813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf9a9-229b-4a26-bccf-6419e2e8e5be None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:11.998300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf9a9-229b-4a26-bccf-6419e2e8e5be None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:11.998767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf9a9-229b-4a26-bccf-6419e2e8e5be None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:12.000291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 687/1358] 10.222.0.58 () {54 vars in 961 bytes} [Mon Oct 3 13:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:12.012685 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f75957-c022-47b9-bd6b-5afac0f54fa0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:12.013331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f75957-c022-47b9-bd6b-5afac0f54fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:12.013805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f75957-c022-47b9-bd6b-5afac0f54fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:12.115845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-54f75957-c022-47b9-bd6b-5afac0f54fa0 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:12.117635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-54f75957-c022-47b9-bd6b-5afac0f54fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:12.175880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:12.259046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 672/1359] 10.222.0.58 () {58 vars in 1058 bytes} [Mon Oct 3 13:23:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:12.268337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf9a9-229b-4a26-bccf-6419e2e8e5be None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:12.272366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0adbf9a9-229b-4a26-bccf-6419e2e8e5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:12.529551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf9a9-229b-4a26-bccf-6419e2e8e5be None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:12.690964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:12.764610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a00a17c-ba72-47cf-93b3-6848d01be6f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:12.765830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00a17c-ba72-47cf-93b3-6848d01be6f6 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:12.766481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00a17c-ba72-47cf-93b3-6848d01be6f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:12.767027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a00a17c-ba72-47cf-93b3-6848d01be6f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:12.770095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9a00a17c-ba72-47cf-93b3-6848d01be6f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:12.943374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00a17c-ba72-47cf-93b3-6848d01be6f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:12.946772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a00a17c-ba72-47cf-93b3-6848d01be6f6 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:12.985091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 688/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:15.800620 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f85f-2db1-4fca-ae18-f0ae83ab42a7 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:15.801637 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f85f-2db1-4fca-ae18-f0ae83ab42a7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:15.801919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f85f-2db1-4fca-ae18-f0ae83ab42a7 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:15.803286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 673/1361] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:15.809061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca795a0-3c91-418c-ab5c-de0caf2766cd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:15.809639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca795a0-3c91-418c-ab5c-de0caf2766cd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:15.809639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca795a0-3c91-418c-ab5c-de0caf2766cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:15.892291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-eca795a0-3c91-418c-ab5c-de0caf2766cd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:15.893586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-eca795a0-3c91-418c-ab5c-de0caf2766cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:15.940402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:16.016558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 689/1362] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:16.335004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3f85f-2db1-4fca-ae18-f0ae83ab42a7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:16.336749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-68b3f85f-2db1-4fca-ae18-f0ae83ab42a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:16.553654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3f85f-2db1-4fca-ae18-f0ae83ab42a7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:16.568613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:16.731132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a44a2-43be-4441-85b5-b871e11c30cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:16.731764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a44a2-43be-4441-85b5-b871e11c30cb service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:16.732051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a44a2-43be-4441-85b5-b871e11c30cb service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:16.732323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a44a2-43be-4441-85b5-b871e11c30cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:16.734091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7b2a44a2-43be-4441-85b5-b871e11c30cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:16.859204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a44a2-43be-4441-85b5-b871e11c30cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:16.861749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a44a2-43be-4441-85b5-b871e11c30cb service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:16.883059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 674/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:19.586029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f271e-91dc-46ad-988f-9f28dac20871 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:19.586555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f271e-91dc-46ad-988f-9f28dac20871 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:19.586555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f271e-91dc-46ad-988f-9f28dac20871 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:19.587877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 690/1364] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:19.593510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc1359-d094-4d8f-8899-109f2d4a40c8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:19.594918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc1359-d094-4d8f-8899-109f2d4a40c8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:19.595181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc1359-d094-4d8f-8899-109f2d4a40c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:19.611842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:19.670487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7adc1359-d094-4d8f-8899-109f2d4a40c8 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:19.671745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7adc1359-d094-4d8f-8899-109f2d4a40c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:19.790633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 675/1365] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:20.085903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962150b0-218a-4a53-84ca-6c68bebb3d90 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:20.086105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962150b0-218a-4a53-84ca-6c68bebb3d90 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:20.086334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962150b0-218a-4a53-84ca-6c68bebb3d90 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:20.088128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 691/1366] 10.222.0.58 () {54 vars in 961 bytes} [Mon Oct 3 13:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:20.094316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddd293-5a39-46eb-b35a-055ce96879f6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:20.094781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddd293-5a39-46eb-b35a-055ce96879f6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:20.095291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddd293-5a39-46eb-b35a-055ce96879f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:20.183615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8bddd293-5a39-46eb-b35a-055ce96879f6 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:20.184849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8bddd293-5a39-46eb-b35a-055ce96879f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:20.228222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:20.319469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 676/1367] 10.222.0.58 () {58 vars in 1058 bytes} [Mon Oct 3 13:23:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:20.327203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962150b0-218a-4a53-84ca-6c68bebb3d90 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:20.329522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-962150b0-218a-4a53-84ca-6c68bebb3d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:20.530271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962150b0-218a-4a53-84ca-6c68bebb3d90 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:20.559273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:20.690857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cbf3a-9c92-4b0a-844a-e60591e416d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:20.691655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cbf3a-9c92-4b0a-844a-e60591e416d9 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:20.692035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cbf3a-9c92-4b0a-844a-e60591e416d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:20.692410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cbf3a-9c92-4b0a-844a-e60591e416d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:20.694335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fd3cbf3a-9c92-4b0a-844a-e60591e416d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:20.832830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cbf3a-9c92-4b0a-844a-e60591e416d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:20.834767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cbf3a-9c92-4b0a-844a-e60591e416d9 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:20.856901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 692/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:20.952781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7447c-3851-4fd2-9ca8-18bcac10e9b6 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:20.953477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7447c-3851-4fd2-9ca8-18bcac10e9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:20.953823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7447c-3851-4fd2-9ca8-18bcac10e9b6 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:20.956394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 677/1369] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:20.964212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74a59-b1e8-43aa-a72c-fa3d481b72fe None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:20.964909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74a59-b1e8-43aa-a72c-fa3d481b72fe None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:20.965425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74a59-b1e8-43aa-a72c-fa3d481b72fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:21.017386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:21.090118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-24d74a59-b1e8-43aa-a72c-fa3d481b72fe None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:21.092399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-24d74a59-b1e8-43aa-a72c-fa3d481b72fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:21.308668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 693/1370] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:21.683409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0d7e7-1f14-4ea6-8587-0f9f2a6c4e2e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:21.683853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0d7e7-1f14-4ea6-8587-0f9f2a6c4e2e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:21.684672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0d7e7-1f14-4ea6-8587-0f9f2a6c4e2e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:21.690093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 678/1371] 10.222.0.58 () {54 vars in 963 bytes} [Mon Oct 3 13:23:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:21.708844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5065e-0eef-4fe8-b27e-3afe63c2edd5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:21.709804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5065e-0eef-4fe8-b27e-3afe63c2edd5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:21.709804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5065e-0eef-4fe8-b27e-3afe63c2edd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:21.820350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-beb5065e-0eef-4fe8-b27e-3afe63c2edd5 None None] MFA Rules not processed for user `8fa402682fd24228a77801edf1bb4d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:21.821812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-beb5065e-0eef-4fe8-b27e-3afe63c2edd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:21.894288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:21.950461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 694/1372] 10.222.0.58 () {58 vars in 1060 bytes} [Mon Oct 3 13:23:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:21.959855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0d7e7-1f14-4ea6-8587-0f9f2a6c4e2e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:21.961438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-02f0d7e7-1f14-4ea6-8587-0f9f2a6c4e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:22.096020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0d7e7-1f14-4ea6-8587-0f9f2a6c4e2e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:22.168027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:22.217433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1c20b-8d34-4e6f-84df-5ff6af109124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:22.218083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c20b-8d34-4e6f-84df-5ff6af109124 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:22.218361 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c20b-8d34-4e6f-84df-5ff6af109124 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:22.218638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c20b-8d34-4e6f-84df-5ff6af109124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:22.220787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-20d1c20b-8d34-4e6f-84df-5ff6af109124 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:22.346350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1c20b-8d34-4e6f-84df-5ff6af109124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:22.347692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1c20b-8d34-4e6f-84df-5ff6af109124 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:22.367049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 679/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:23:23.262407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdef3a-e1c2-4c3b-9a8d-0ea560be8bac None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:23.262973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdef3a-e1c2-4c3b-9a8d-0ea560be8bac None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:23.262973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdef3a-e1c2-4c3b-9a8d-0ea560be8bac None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:23.264731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 695/1374] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:23.272950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f183-e777-4454-8428-9b37197dac15 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:23.273551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f183-e777-4454-8428-9b37197dac15 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:23.274056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192f183-e777-4454-8428-9b37197dac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:23.347763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1192f183-e777-4454-8428-9b37197dac15 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:23.349009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1192f183-e777-4454-8428-9b37197dac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:23.371779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:23.470392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 680/1375] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:23:23.767206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdef3a-e1c2-4c3b-9a8d-0ea560be8bac None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:23.768755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d8cdef3a-e1c2-4c3b-9a8d-0ea560be8bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:23.946754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdef3a-e1c2-4c3b-9a8d-0ea560be8bac None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:24.013992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:24.099843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd8a1d-b3c4-409a-9e52-cc495c649775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:24.100462 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd8a1d-b3c4-409a-9e52-cc495c649775 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:24.100747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd8a1d-b3c4-409a-9e52-cc495c649775 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:24.101026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd8a1d-b3c4-409a-9e52-cc495c649775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:24.102766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7ecd8a1d-b3c4-409a-9e52-cc495c649775 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:24.214331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd8a1d-b3c4-409a-9e52-cc495c649775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:24.216016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd8a1d-b3c4-409a-9e52-cc495c649775 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:24.238551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 696/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:26.910502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495826cc-3224-4be2-ba64-037f981d1975 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:26.911350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495826cc-3224-4be2-ba64-037f981d1975 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:26.911535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495826cc-3224-4be2-ba64-037f981d1975 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:26.914296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 681/1377] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:26.923884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4463e-2581-4f66-be96-d3c95966eaea None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:26.924756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4463e-2581-4f66-be96-d3c95966eaea None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:26.925138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4463e-2581-4f66-be96-d3c95966eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:27.049693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b5a4463e-2581-4f66-be96-d3c95966eaea None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:27.051627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b5a4463e-2581-4f66-be96-d3c95966eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:27.064128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:27.245624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 697/1378] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:27.691017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495826cc-3224-4be2-ba64-037f981d1975 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:27.692629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-495826cc-3224-4be2-ba64-037f981d1975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:27.938842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495826cc-3224-4be2-ba64-037f981d1975 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:27.941583 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:28.103178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709723d3-52ff-43d6-a08d-941d43895293 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:28.103801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709723d3-52ff-43d6-a08d-941d43895293 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:28.104089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709723d3-52ff-43d6-a08d-941d43895293 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:28.104358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709723d3-52ff-43d6-a08d-941d43895293 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:28.106274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-709723d3-52ff-43d6-a08d-941d43895293 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:28.220913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709723d3-52ff-43d6-a08d-941d43895293 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:28.227192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709723d3-52ff-43d6-a08d-941d43895293 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:28.263356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 682/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:30.702891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d0132-e83b-475d-8997-10d418e5d2cf None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:30.704087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d0132-e83b-475d-8997-10d418e5d2cf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:30.704087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d0132-e83b-475d-8997-10d418e5d2cf None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:30.705487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 698/1380] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:23:30.711443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c525-810a-49f5-ae71-428f5f8ae910 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:30.712359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c525-810a-49f5-ae71-428f5f8ae910 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:30.712763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272c525-810a-49f5-ae71-428f5f8ae910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:30.726220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:30.792831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c272c525-810a-49f5-ae71-428f5f8ae910 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:30.794286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c272c525-810a-49f5-ae71-428f5f8ae910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:30.914250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 683/1381] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:31.418616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d0132-e83b-475d-8997-10d418e5d2cf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:31.420663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2f8d0132-e83b-475d-8997-10d418e5d2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:31.615341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:31.629386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d0132-e83b-475d-8997-10d418e5d2cf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:31.783516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff8bcd-9270-411d-9da9-472e250fb62e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:31.784644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8bcd-9270-411d-9da9-472e250fb62e service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:31.784644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8bcd-9270-411d-9da9-472e250fb62e service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:31.785057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8bcd-9270-411d-9da9-472e250fb62e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:31.786550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e7ff8bcd-9270-411d-9da9-472e250fb62e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:32.053508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff8bcd-9270-411d-9da9-472e250fb62e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:32.056895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff8bcd-9270-411d-9da9-472e250fb62e service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:32.082547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:32.097123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 699/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:34.832895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb453c-e0ec-439d-846e-869957c251a5 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:34.833590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb453c-e0ec-439d-846e-869957c251a5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:34.833590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb453c-e0ec-439d-846e-869957c251a5 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:34.834919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 684/1383] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:23:34.840993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb75f4-26a6-4262-91f1-af7ac3fd13e6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:34.841643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb75f4-26a6-4262-91f1-af7ac3fd13e6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:34.841643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb75f4-26a6-4262-91f1-af7ac3fd13e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:34.936487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c7eb75f4-26a6-4262-91f1-af7ac3fd13e6 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:34.938615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c7eb75f4-26a6-4262-91f1-af7ac3fd13e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:35.087732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 700/1384] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:35.461668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb453c-e0ec-439d-846e-869957c251a5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:35.463216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-44fb453c-e0ec-439d-846e-869957c251a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:35.577695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:35.675898 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb453c-e0ec-439d-846e-869957c251a5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:35.849015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff261ff9-ee02-4953-98cb-987bdeb4151b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:35.849933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261ff9-ee02-4953-98cb-987bdeb4151b service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:35.850258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261ff9-ee02-4953-98cb-987bdeb4151b service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:35.850626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff261ff9-ee02-4953-98cb-987bdeb4151b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:35.853066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ff261ff9-ee02-4953-98cb-987bdeb4151b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:35.930855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:35.986297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff261ff9-ee02-4953-98cb-987bdeb4151b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:35.988600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff261ff9-ee02-4953-98cb-987bdeb4151b service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:36.017732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 685/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:38.479215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f840f74-fde9-4167-ae86-bc6bb2baa55e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:38.479800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f840f74-fde9-4167-ae86-bc6bb2baa55e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:38.479800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f840f74-fde9-4167-ae86-bc6bb2baa55e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:38.481173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 701/1386] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:38.488011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb95f-5936-4ff2-96a0-53c89941054b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:38.488546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb95f-5936-4ff2-96a0-53c89941054b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:38.488792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2cb95f-5936-4ff2-96a0-53c89941054b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:38.576724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0d2cb95f-5936-4ff2-96a0-53c89941054b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:38.578066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0d2cb95f-5936-4ff2-96a0-53c89941054b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:38.705445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 686/1387] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:38.996969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64171f3-080f-4bc1-aa85-a1670a00e3d5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:38.997145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64171f3-080f-4bc1-aa85-a1670a00e3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:38.997258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64171f3-080f-4bc1-aa85-a1670a00e3d5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:38.998656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 702/1388] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:39.004492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558dd63-4fe9-4262-9ab6-040fa8d96e43 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:39.004777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558dd63-4fe9-4262-9ab6-040fa8d96e43 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:39.005050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558dd63-4fe9-4262-9ab6-040fa8d96e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:39.007722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64171f3-080f-4bc1-aa85-a1670a00e3d5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:39.009263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d64171f3-080f-4bc1-aa85-a1670a00e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:39.047513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:39.084386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0558dd63-4fe9-4262-9ab6-040fa8d96e43 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:39.085695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0558dd63-4fe9-4262-9ab6-040fa8d96e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:39.113293 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:39.194819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64171f3-080f-4bc1-aa85-a1670a00e3d5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:39.195393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 687/1389] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:23:39.363101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5734af-3d10-4285-938c-f446ff2a51af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:39.363716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5734af-3d10-4285-938c-f446ff2a51af service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:39.364064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5734af-3d10-4285-938c-f446ff2a51af service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:39.364362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5734af-3d10-4285-938c-f446ff2a51af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:39.366070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-af5734af-3d10-4285-938c-f446ff2a51af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:39.412402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:39.469033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5734af-3d10-4285-938c-f446ff2a51af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:39.470786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5734af-3d10-4285-938c-f446ff2a51af service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:39.493331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 703/1390] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:23:39.515437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7ff6-a633-4530-a303-bbb1ca4cf3a9 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:39.516757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7ff6-a633-4530-a303-bbb1ca4cf3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:39.516757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e7ff6-a633-4530-a303-bbb1ca4cf3a9 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:39.518282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 688/1391] 10.222.0.58 () {54 vars in 962 bytes} [Mon Oct 3 13:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:39.524902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830b490-0bc1-4b41-a30e-9f3ae6402855 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:39.525316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830b490-0bc1-4b41-a30e-9f3ae6402855 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:39.525820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830b490-0bc1-4b41-a30e-9f3ae6402855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:39.635777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0830b490-0bc1-4b41-a30e-9f3ae6402855 None None] MFA Rules not processed for user `464190e0ccaf4f82942f518df1bf6c03`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:39.638133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0830b490-0bc1-4b41-a30e-9f3ae6402855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:39.782096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 704/1392] 10.222.0.58 () {58 vars in 1059 bytes} [Mon Oct 3 13:23:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:39.791803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7ff6-a633-4530-a303-bbb1ca4cf3a9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:39.793161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-636e7ff6-a633-4530-a303-bbb1ca4cf3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:39.934749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e7ff6-a633-4530-a303-bbb1ca4cf3a9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:39.943707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:40.061184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6393e3-2469-4d1f-8e50-50bcb01efb07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '464190e0ccaf4f82942f518df1bf6c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:40.061767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6393e3-2469-4d1f-8e50-50bcb01efb07 service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:40.062047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6393e3-2469-4d1f-8e50-50bcb01efb07 service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:40.062338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6393e3-2469-4d1f-8e50-50bcb01efb07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:40.064060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2c6393e3-2469-4d1f-8e50-50bcb01efb07 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:40.195237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6393e3-2469-4d1f-8e50-50bcb01efb07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:40.196692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6393e3-2469-4d1f-8e50-50bcb01efb07 service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:40.217112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 689/1393] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:41.034043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603ad5e-dab5-42ec-ac00-5435d0821b34 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:41.034590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603ad5e-dab5-42ec-ac00-5435d0821b34 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:41.034590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603ad5e-dab5-42ec-ac00-5435d0821b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:41.073326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:41.107878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1603ad5e-dab5-42ec-ac00-5435d0821b34 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:41.109092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1603ad5e-dab5-42ec-ac00-5435d0821b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:41.220398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 705/1394] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:23:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:41.272100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65697358-9334-4c48-8410-c0e6e2d33b83 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:41.272100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65697358-9334-4c48-8410-c0e6e2d33b83 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:41.272100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65697358-9334-4c48-8410-c0e6e2d33b83 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:41.273689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 690/1395] 10.222.0.58 () {54 vars in 943 bytes} [Mon Oct 3 13:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:41.280164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8038-2f46-4017-b997-a48610a4ead2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:41.280455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8038-2f46-4017-b997-a48610a4ead2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:41.280733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8038-2f46-4017-b997-a48610a4ead2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:41.375038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3fdf8038-2f46-4017-b997-a48610a4ead2 None None] MFA Rules not processed for user `4ba4c42b47794c9ba5fc6ee557e2cfc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:41.376854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3fdf8038-2f46-4017-b997-a48610a4ead2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:41.443423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:41.526426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 706/1396] 10.222.0.58 () {58 vars in 1040 bytes} [Mon Oct 3 13:23:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:41.540060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65697358-9334-4c48-8410-c0e6e2d33b83 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:41.542451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-65697358-9334-4c48-8410-c0e6e2d33b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:41.590619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:41.698980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65697358-9334-4c48-8410-c0e6e2d33b83 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:41.788089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5534c4-f62f-487b-bad7-7a07dce17172 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:41.788284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5534c4-f62f-487b-bad7-7a07dce17172 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:41.788570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5534c4-f62f-487b-bad7-7a07dce17172 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:41.790157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 707/1397] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:23:41.795205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c44a7-d030-4c26-aa76-c007b896639b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:41.795490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c44a7-d030-4c26-aa76-c007b896639b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:41.795770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c44a7-d030-4c26-aa76-c007b896639b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:41.811145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:41.811766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:41.812045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:41.812445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:41.814886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:41.837706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:41.867652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8b3c44a7-d030-4c26-aa76-c007b896639b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:41.868894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8b3c44a7-d030-4c26-aa76-c007b896639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:41.904083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:41.905719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:41.915303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 691/1398] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:23:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:23:42.002630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 708/1399] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:23:42.293804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:42.295026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:42.411172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2881-35c5-45bb-b6e0-053efe500392 service swift] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:42.492226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:42.558806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28562a01-aad1-4b88-b659-e4eda90f6da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:42.559348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28562a01-aad1-4b88-b659-e4eda90f6da9 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:42.559632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28562a01-aad1-4b88-b659-e4eda90f6da9 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:42.559907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28562a01-aad1-4b88-b659-e4eda90f6da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:42.561590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-28562a01-aad1-4b88-b659-e4eda90f6da9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:42.671275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28562a01-aad1-4b88-b659-e4eda90f6da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:42.673606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28562a01-aad1-4b88-b659-e4eda90f6da9 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:42.695257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 692/1400] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:44.997353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf4af-02de-4b37-9b2f-d69d125678fe None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:44.997353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf4af-02de-4b37-9b2f-d69d125678fe None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:44.997353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf4af-02de-4b37-9b2f-d69d125678fe None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:45.002286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d7da-00da-4fd7-8259-e9152a8343c7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:45.003347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d7da-00da-4fd7-8259-e9152a8343c7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:45.003347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d7da-00da-4fd7-8259-e9152a8343c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:45.007473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 709/1401] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:23:45.108657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6d36d7da-00da-4fd7-8259-e9152a8343c7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:45.110024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6d36d7da-00da-4fd7-8259-e9152a8343c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:45.144225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:45.283574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 693/1402] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:45.595241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bf4af-02de-4b37-9b2f-d69d125678fe None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:45.596807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-803bf4af-02de-4b37-9b2f-d69d125678fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:45.638717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:45.808855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bf4af-02de-4b37-9b2f-d69d125678fe None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:45.948212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:45.970269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e2e89-a905-4825-bc31-195398970134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:45.971440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e2e89-a905-4825-bc31-195398970134 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:45.971960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e2e89-a905-4825-bc31-195398970134 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:45.972547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e2e89-a905-4825-bc31-195398970134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:45.975965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6b0e2e89-a905-4825-bc31-195398970134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:46.141367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e2e89-a905-4825-bc31-195398970134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:46.144592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e2e89-a905-4825-bc31-195398970134 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:46.184826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 710/1403] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:23:48.451968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43c9e-7c8f-4958-a442-cfd70c025510 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:48.452530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43c9e-7c8f-4958-a442-cfd70c025510 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:48.452530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43c9e-7c8f-4958-a442-cfd70c025510 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:48.455032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 694/1404] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:48.460586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb4e5-d742-4551-9a62-d6ffbc5b05bc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:48.461064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb4e5-d742-4551-9a62-d6ffbc5b05bc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:48.461225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb4e5-d742-4551-9a62-d6ffbc5b05bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:48.542924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-14fbb4e5-d742-4551-9a62-d6ffbc5b05bc None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:48.544248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-14fbb4e5-d742-4551-9a62-d6ffbc5b05bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:48.622850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:48.666544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 711/1405] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:48.963962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe43c9e-7c8f-4958-a442-cfd70c025510 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:48.965521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6fe43c9e-7c8f-4958-a442-cfd70c025510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:49.136939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe43c9e-7c8f-4958-a442-cfd70c025510 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:49.150294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:49.275718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17efb0-d6c7-4010-8d75-34e983646c57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:49.276341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17efb0-d6c7-4010-8d75-34e983646c57 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:49.276625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17efb0-d6c7-4010-8d75-34e983646c57 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:49.276902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17efb0-d6c7-4010-8d75-34e983646c57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:49.283803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0b17efb0-d6c7-4010-8d75-34e983646c57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:49.388053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17efb0-d6c7-4010-8d75-34e983646c57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:49.389722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17efb0-d6c7-4010-8d75-34e983646c57 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:49.413833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 695/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:23:51.736371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749555-00a5-498b-83f2-f97144158f7b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:51.736913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749555-00a5-498b-83f2-f97144158f7b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:51.736913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749555-00a5-498b-83f2-f97144158f7b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:51.738316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 712/1407] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:23:51.743734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7230b9-9bf2-4c79-a7d2-c447acfd4a0b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:51.744565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7230b9-9bf2-4c79-a7d2-c447acfd4a0b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:51.744860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7230b9-9bf2-4c79-a7d2-c447acfd4a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:51.770437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:51.824158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fe7230b9-9bf2-4c79-a7d2-c447acfd4a0b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:23:51.825385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fe7230b9-9bf2-4c79-a7d2-c447acfd4a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:51.945427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 696/1408] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:23:52.241004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c749555-00a5-498b-83f2-f97144158f7b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:23:52.242673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4c749555-00a5-498b-83f2-f97144158f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:52.417813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c749555-00a5-498b-83f2-f97144158f7b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:23:52.479818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:23:52.557228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b8fe6-ec74-4f08-bbb9-b517f24f6179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:23:52.557891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b8fe6-ec74-4f08-bbb9-b517f24f6179 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:23:52.558173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b8fe6-ec74-4f08-bbb9-b517f24f6179 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:23:52.558453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b8fe6-ec74-4f08-bbb9-b517f24f6179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:23:52.560106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-386b8fe6-ec74-4f08-bbb9-b517f24f6179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:23:52.669238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b8fe6-ec74-4f08-bbb9-b517f24f6179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:23:52.671322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b8fe6-ec74-4f08-bbb9-b517f24f6179 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:23:52.695344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 713/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:01.943750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=58 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:02.692652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=76 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:05.217608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6ddde-e7d4-4c9d-a7c9-cca562fa9d6a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:05.218442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6ddde-e7d4-4c9d-a7c9-cca562fa9d6a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:05.218442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6ddde-e7d4-4c9d-a7c9-cca562fa9d6a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:05.219984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 697/1410] 10.222.0.58 () {54 vars in 918 bytes} [Mon Oct 3 13:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:05.225431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961636-32d5-473d-8c87-ca44fed2d856 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:05.225948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961636-32d5-473d-8c87-ca44fed2d856 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:05.226042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb961636-32d5-473d-8c87-ca44fed2d856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:05.307282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-eb961636-32d5-473d-8c87-ca44fed2d856 None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:05.308734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-eb961636-32d5-473d-8c87-ca44fed2d856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:05.421056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 714/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Oct 3 13:24:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:05.439593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccc144-3ad3-4504-b6ce-5b1c71ca0c5f None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:05.439871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccc144-3ad3-4504-b6ce-5b1c71ca0c5f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:05.440141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dccc144-3ad3-4504-b6ce-5b1c71ca0c5f None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:05.441559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 698/1412] 10.222.0.58 () {54 vars in 918 bytes} [Mon Oct 3 13:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:24:05.447307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5bc6b-4730-47e8-a871-9c715cce105b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:05.447600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5bc6b-4730-47e8-a871-9c715cce105b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:05.447884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5bc6b-4730-47e8-a871-9c715cce105b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:05.529662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:05.534947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2bc5bc6b-4730-47e8-a871-9c715cce105b None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:05.536622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2bc5bc6b-4730-47e8-a871-9c715cce105b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:05.691351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 715/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Oct 3 13:24:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:08.409889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71646ab9-a5a7-4a81-8e66-1b27cb8fc867 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:08.410475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71646ab9-a5a7-4a81-8e66-1b27cb8fc867 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:08.410475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71646ab9-a5a7-4a81-8e66-1b27cb8fc867 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:08.411801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:08.417206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6da33-62fc-43cb-9b1a-ea8f3cf5aeda None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:08.417672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6da33-62fc-43cb-9b1a-ea8f3cf5aeda None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:08.417789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6da33-62fc-43cb-9b1a-ea8f3cf5aeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:08.488494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-53a6da33-62fc-43cb-9b1a-ea8f3cf5aeda None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:08.489779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-53a6da33-62fc-43cb-9b1a-ea8f3cf5aeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:08.551926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:08.602197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:08.895798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71646ab9-a5a7-4a81-8e66-1b27cb8fc867 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:08.897322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-71646ab9-a5a7-4a81-8e66-1b27cb8fc867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:09.053896 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71646ab9-a5a7-4a81-8e66-1b27cb8fc867 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:09.130989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:09.173926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbb8af-855d-4108-a3be-591d1746bb6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:09.174550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbb8af-855d-4108-a3be-591d1746bb6a service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:09.175212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbb8af-855d-4108-a3be-591d1746bb6a service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:09.175212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbb8af-855d-4108-a3be-591d1746bb6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:09.176741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-93fbb8af-855d-4108-a3be-591d1746bb6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:09.271850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbb8af-855d-4108-a3be-591d1746bb6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:09.273529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbb8af-855d-4108-a3be-591d1746bb6a service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:09.292063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 700/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:10.473927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffdb97-deb1-4048-8f8f-da2ef00c9553 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:10.474515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffdb97-deb1-4048-8f8f-da2ef00c9553 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:10.474515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffdb97-deb1-4048-8f8f-da2ef00c9553 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:10.475854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:10.482586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c70473-b6a0-4172-bbbc-c00eba1bf26e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:10.483052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c70473-b6a0-4172-bbbc-c00eba1bf26e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:10.483164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c70473-b6a0-4172-bbbc-c00eba1bf26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:10.562679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-62c70473-b6a0-4172-bbbc-c00eba1bf26e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:10.563992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-62c70473-b6a0-4172-bbbc-c00eba1bf26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:10.592371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:10.668214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:10.825172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffdb97-deb1-4048-8f8f-da2ef00c9553 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:10.826738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-caffdb97-deb1-4048-8f8f-da2ef00c9553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:10.976847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffdb97-deb1-4048-8f8f-da2ef00c9553 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:11.019335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:11.096011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec0a51-3af9-4781-a98c-1c6c3817106a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:11.096738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec0a51-3af9-4781-a98c-1c6c3817106a service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:11.097051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec0a51-3af9-4781-a98c-1c6c3817106a service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:11.097325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec0a51-3af9-4781-a98c-1c6c3817106a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:11.099033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b6ec0a51-3af9-4781-a98c-1c6c3817106a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:11.192807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec0a51-3af9-4781-a98c-1c6c3817106a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:11.194752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec0a51-3af9-4781-a98c-1c6c3817106a service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:11.213654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 718/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:12.369957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30d94a-090e-4e09-9340-1a4b418a3f1c None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:12.370536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30d94a-090e-4e09-9340-1a4b418a3f1c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:12.370536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30d94a-090e-4e09-9340-1a4b418a3f1c None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:12.371842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:12.379671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9dc25-38ef-4525-ae0e-7feea7a8084e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:12.380219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9dc25-38ef-4525-ae0e-7feea7a8084e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:12.380219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b9dc25-38ef-4525-ae0e-7feea7a8084e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:12.437782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:12.464730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a1b9dc25-38ef-4525-ae0e-7feea7a8084e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:12.466040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a1b9dc25-38ef-4525-ae0e-7feea7a8084e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:12.578894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:12.738891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30d94a-090e-4e09-9340-1a4b418a3f1c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:12.740471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-db30d94a-090e-4e09-9340-1a4b418a3f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:12.898992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30d94a-090e-4e09-9340-1a4b418a3f1c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:12.911744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:13.019365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923049d-3962-45d9-8ec3-a804820c4530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:13.019880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923049d-3962-45d9-8ec3-a804820c4530 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:13.020157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923049d-3962-45d9-8ec3-a804820c4530 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:13.020427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923049d-3962-45d9-8ec3-a804820c4530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:13.022062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7923049d-3962-45d9-8ec3-a804820c4530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:13.115005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923049d-3962-45d9-8ec3-a804820c4530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:13.116626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923049d-3962-45d9-8ec3-a804820c4530 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:13.135766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 703/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:14.285092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a10b-27a8-4989-a9cb-4876e5c65498 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:14.285666 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a10b-27a8-4989-a9cb-4876e5c65498 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:14.285764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a10b-27a8-4989-a9cb-4876e5c65498 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:14.287063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:14.293654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b77b1-bec2-4b2d-b7f2-aba01987b58e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:14.294143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b77b1-bec2-4b2d-b7f2-aba01987b58e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:14.294241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b77b1-bec2-4b2d-b7f2-aba01987b58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:14.313848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:14.374138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-eb9b77b1-bec2-4b2d-b7f2-aba01987b58e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:14.375610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-eb9b77b1-bec2-4b2d-b7f2-aba01987b58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:14.486315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:14.639602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a10b-27a8-4989-a9cb-4876e5c65498 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:14.641668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5348a10b-27a8-4989-a9cb-4876e5c65498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:14.768550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:14.810533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a10b-27a8-4989-a9cb-4876e5c65498 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:14.932925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb5056-b3eb-4fde-a3ce-f8801cd839d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:14.933849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb5056-b3eb-4fde-a3ce-f8801cd839d3 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:14.934140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb5056-b3eb-4fde-a3ce-f8801cd839d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:14.934415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb5056-b3eb-4fde-a3ce-f8801cd839d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:14.936179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cbbb5056-b3eb-4fde-a3ce-f8801cd839d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:15.020001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:15.034514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb5056-b3eb-4fde-a3ce-f8801cd839d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:15.036134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb5056-b3eb-4fde-a3ce-f8801cd839d3 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:15.055099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 721/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:16.195293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b234a43-3f68-4a36-a0d0-0b67f654ccc6 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:16.195879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b234a43-3f68-4a36-a0d0-0b67f654ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:16.195879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b234a43-3f68-4a36-a0d0-0b67f654ccc6 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:16.197221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:16.205110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4380d-628a-4de2-b2b7-26ba9f95eb0d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:16.205696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4380d-628a-4de2-b2b7-26ba9f95eb0d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:16.205831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4380d-628a-4de2-b2b7-26ba9f95eb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:16.294225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1be4380d-628a-4de2-b2b7-26ba9f95eb0d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:16.295587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1be4380d-628a-4de2-b2b7-26ba9f95eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:16.402843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:16.563316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b234a43-3f68-4a36-a0d0-0b67f654ccc6 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:16.564850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4b234a43-3f68-4a36-a0d0-0b67f654ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:16.663793 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:16.726164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b234a43-3f68-4a36-a0d0-0b67f654ccc6 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:16.846842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356021e-dceb-4385-8c52-f218f10f05a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:16.847373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356021e-dceb-4385-8c52-f218f10f05a5 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:16.847651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356021e-dceb-4385-8c52-f218f10f05a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:16.847922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356021e-dceb-4385-8c52-f218f10f05a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:16.849559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b356021e-dceb-4385-8c52-f218f10f05a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:16.900495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:16.949462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356021e-dceb-4385-8c52-f218f10f05a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:16.951100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356021e-dceb-4385-8c52-f218f10f05a5 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:16.970333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 706/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:18.053366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e9a1e-d2ac-4643-9331-843f12bcef99 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:18.054005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e9a1e-d2ac-4643-9331-843f12bcef99 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:18.054005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e9a1e-d2ac-4643-9331-843f12bcef99 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:18.055317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:18.061879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676d9e7-b187-4c3e-9ed7-beed77ac4cfa None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:18.062367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676d9e7-b187-4c3e-9ed7-beed77ac4cfa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:18.062496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676d9e7-b187-4c3e-9ed7-beed77ac4cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:18.142853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3676d9e7-b187-4c3e-9ed7-beed77ac4cfa None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:18.144112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3676d9e7-b187-4c3e-9ed7-beed77ac4cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:18.247977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:18.403948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e9a1e-d2ac-4643-9331-843f12bcef99 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:18.406223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-742e9a1e-d2ac-4643-9331-843f12bcef99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:18.462887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:18.593017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e9a1e-d2ac-4643-9331-843f12bcef99 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:18.716205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d2771-1c47-45d9-a00d-44a04c177206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:18.716808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d2771-1c47-45d9-a00d-44a04c177206 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:18.717092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d2771-1c47-45d9-a00d-44a04c177206 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:18.717368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d2771-1c47-45d9-a00d-44a04c177206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:18.719437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-268d2771-1c47-45d9-a00d-44a04c177206 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:18.727209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:18.814160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d2771-1c47-45d9-a00d-44a04c177206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:18.815780 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d2771-1c47-45d9-a00d-44a04c177206 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:18.833897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 724/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:19.926818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0128b-cec8-4845-af3d-2194219af8de None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:19.927407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0128b-cec8-4845-af3d-2194219af8de None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:19.927407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0128b-cec8-4845-af3d-2194219af8de None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:19.928738 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:19.935511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12857c12-f2cf-4af9-a5b2-9f938d44dbb7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:19.936043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12857c12-f2cf-4af9-a5b2-9f938d44dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:19.936152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12857c12-f2cf-4af9-a5b2-9f938d44dbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:20.014494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-12857c12-f2cf-4af9-a5b2-9f938d44dbb7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:20.016637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-12857c12-f2cf-4af9-a5b2-9f938d44dbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:20.088902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:20.121173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:20.275766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0128b-cec8-4845-af3d-2194219af8de None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:20.277348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bed0128b-cec8-4845-af3d-2194219af8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:20.297232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:20.446884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0128b-cec8-4845-af3d-2194219af8de None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:20.537811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:20.564727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da1db4-3d9f-49e6-92f4-b8727edb2deb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:20.565336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1db4-3d9f-49e6-92f4-b8727edb2deb service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:20.565658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1db4-3d9f-49e6-92f4-b8727edb2deb service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:20.565934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da1db4-3d9f-49e6-92f4-b8727edb2deb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:20.567555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d8da1db4-3d9f-49e6-92f4-b8727edb2deb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:20.662262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da1db4-3d9f-49e6-92f4-b8727edb2deb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:20.663895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da1db4-3d9f-49e6-92f4-b8727edb2deb service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:20.682188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 709/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:21.759039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c80c4-2527-4f2e-93f1-08d184f7122f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:21.759620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c80c4-2527-4f2e-93f1-08d184f7122f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:21.759620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c80c4-2527-4f2e-93f1-08d184f7122f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:21.760951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:21.768315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab32c5-99e1-4134-9464-7d1c12a0e16e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:21.768865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab32c5-99e1-4134-9464-7d1c12a0e16e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:21.768865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab32c5-99e1-4134-9464-7d1c12a0e16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:21.847969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7aab32c5-99e1-4134-9464-7d1c12a0e16e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:21.849203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7aab32c5-99e1-4134-9464-7d1c12a0e16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:21.892427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:21.950219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:22.101123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c80c4-2527-4f2e-93f1-08d184f7122f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:22.102618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-599c80c4-2527-4f2e-93f1-08d184f7122f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:22.260683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c80c4-2527-4f2e-93f1-08d184f7122f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:22.319024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:22.385482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbada671-8be8-413d-9425-7f07b7ab3bf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:22.385996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbada671-8be8-413d-9425-7f07b7ab3bf4 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:22.386278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbada671-8be8-413d-9425-7f07b7ab3bf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:22.386544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbada671-8be8-413d-9425-7f07b7ab3bf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:22.388142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cbada671-8be8-413d-9425-7f07b7ab3bf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:22.482089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbada671-8be8-413d-9425-7f07b7ab3bf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:22.484272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbada671-8be8-413d-9425-7f07b7ab3bf4 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:22.504161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 727/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:23.599236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222d406-4c74-498f-b133-4986a34ea718 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:23.599758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222d406-4c74-498f-b133-4986a34ea718 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:23.599758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222d406-4c74-498f-b133-4986a34ea718 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:23.601132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:23.607626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475910-1100-4452-b23e-25b8d0b43fd3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:23.608101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475910-1100-4452-b23e-25b8d0b43fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:23.608199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475910-1100-4452-b23e-25b8d0b43fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:23.686741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-04475910-1100-4452-b23e-25b8d0b43fd3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:23.687985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-04475910-1100-4452-b23e-25b8d0b43fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:23.690068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:23.788256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:23.938055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222d406-4c74-498f-b133-4986a34ea718 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:23.939539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1222d406-4c74-498f-b133-4986a34ea718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:24.087122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222d406-4c74-498f-b133-4986a34ea718 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:24.114732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:24.204902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8bbcf-ab2e-4a5f-9543-ce3d3020b3d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:24.205429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8bbcf-ab2e-4a5f-9543-ce3d3020b3d0 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:24.205746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8bbcf-ab2e-4a5f-9543-ce3d3020b3d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:24.206019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8bbcf-ab2e-4a5f-9543-ce3d3020b3d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:24.207643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dcb8bbcf-ab2e-4a5f-9543-ce3d3020b3d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:24.301538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8bbcf-ab2e-4a5f-9543-ce3d3020b3d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:24.303158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8bbcf-ab2e-4a5f-9543-ce3d3020b3d0 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:24.321702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 712/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:26.454762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf9140-91fe-4beb-afac-437de8715ca8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:26.455332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf9140-91fe-4beb-afac-437de8715ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:26.455332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf9140-91fe-4beb-afac-437de8715ca8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:26.456769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:26.462170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc77e-ba5f-4a41-bc43-aebe9610093e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:26.462625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc77e-ba5f-4a41-bc43-aebe9610093e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:26.462729 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc77e-ba5f-4a41-bc43-aebe9610093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:26.510376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:26.550271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2f9cc77e-ba5f-4a41-bc43-aebe9610093e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:26.551522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2f9cc77e-ba5f-4a41-bc43-aebe9610093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:26.653716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:26.944710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf9140-91fe-4beb-afac-437de8715ca8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:26.946298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-93cf9140-91fe-4beb-afac-437de8715ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:27.052359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf9140-91fe-4beb-afac-437de8715ca8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:27.085441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:27.128824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1025e-4db8-489b-80b5-5731f77430cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:27.129443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1025e-4db8-489b-80b5-5731f77430cb service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:27.129832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1025e-4db8-489b-80b5-5731f77430cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:27.130115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1025e-4db8-489b-80b5-5731f77430cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:27.131746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-26d1025e-4db8-489b-80b5-5731f77430cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:27.243206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1025e-4db8-489b-80b5-5731f77430cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:27.244553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1025e-4db8-489b-80b5-5731f77430cb service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:27.263882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 730/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:29.626918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d77c76-0c98-4ae1-a2a1-31d9c162635e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:29.627551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d77c76-0c98-4ae1-a2a1-31d9c162635e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:29.627551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d77c76-0c98-4ae1-a2a1-31d9c162635e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:29.628997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:29.634548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d12523-0cfc-4638-9dc2-4fa0777b6df6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:29.635044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d12523-0cfc-4638-9dc2-4fa0777b6df6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:29.635151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d12523-0cfc-4638-9dc2-4fa0777b6df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:29.705220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a1d12523-0cfc-4638-9dc2-4fa0777b6df6 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:29.706488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a1d12523-0cfc-4638-9dc2-4fa0777b6df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:29.720950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:29.810329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:30.101408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d77c76-0c98-4ae1-a2a1-31d9c162635e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:30.102999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c0d77c76-0c98-4ae1-a2a1-31d9c162635e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:30.212484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d77c76-0c98-4ae1-a2a1-31d9c162635e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:30.224702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:30.287740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa93b9e-555b-46cb-97da-24526b64bb54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:30.288363 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa93b9e-555b-46cb-97da-24526b64bb54 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:30.288644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa93b9e-555b-46cb-97da-24526b64bb54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:30.288916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa93b9e-555b-46cb-97da-24526b64bb54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:30.290556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4fa93b9e-555b-46cb-97da-24526b64bb54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:30.405938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa93b9e-555b-46cb-97da-24526b64bb54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:30.407190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa93b9e-555b-46cb-97da-24526b64bb54 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:30.425716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 715/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:32.963529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e63ffc-813d-4879-a9a0-331e15bc85cd None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:32.963960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e63ffc-813d-4879-a9a0-331e15bc85cd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:32.964103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e63ffc-813d-4879-a9a0-331e15bc85cd None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:32.965493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:32.971620 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdb3c9-6f3e-4057-8bcd-1f3c32d04554 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:32.972086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdb3c9-6f3e-4057-8bcd-1f3c32d04554 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:32.972181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdb3c9-6f3e-4057-8bcd-1f3c32d04554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:33.021803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:33.040643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ccbdb3c9-6f3e-4057-8bcd-1f3c32d04554 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:33.041891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ccbdb3c9-6f3e-4057-8bcd-1f3c32d04554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:33.138748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:33.426572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e63ffc-813d-4879-a9a0-331e15bc85cd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:33.428105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-52e63ffc-813d-4879-a9a0-331e15bc85cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:33.535640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e63ffc-813d-4879-a9a0-331e15bc85cd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:33.578068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:33.622943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378c7c9-f10b-4c4d-b877-1abb6a0632ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:33.623555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c7c9-f10b-4c4d-b877-1abb6a0632ca service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:33.623939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c7c9-f10b-4c4d-b877-1abb6a0632ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:33.624257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c7c9-f10b-4c4d-b877-1abb6a0632ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:33.625955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b378c7c9-f10b-4c4d-b877-1abb6a0632ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:33.739926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378c7c9-f10b-4c4d-b877-1abb6a0632ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:33.741371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378c7c9-f10b-4c4d-b877-1abb6a0632ca service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:33.760384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 733/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:36.140865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193af0d-9fc8-4155-9346-ae7742bb887b None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:36.141537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193af0d-9fc8-4155-9346-ae7742bb887b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:36.141537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193af0d-9fc8-4155-9346-ae7742bb887b None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:36.142813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:36.148315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201495b8-7dee-4599-8f48-b18483f6237f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:36.148758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201495b8-7dee-4599-8f48-b18483f6237f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:36.148881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201495b8-7dee-4599-8f48-b18483f6237f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:36.221297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-201495b8-7dee-4599-8f48-b18483f6237f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:36.222579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-201495b8-7dee-4599-8f48-b18483f6237f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:36.243845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:36.323072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:36.613077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193af0d-9fc8-4155-9346-ae7742bb887b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:36.614662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c193af0d-9fc8-4155-9346-ae7742bb887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:36.722326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193af0d-9fc8-4155-9346-ae7742bb887b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:36.747516 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:36.802985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a754c-6b17-4030-9f89-e7c3491cb15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:36.803606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a754c-6b17-4030-9f89-e7c3491cb15b service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:36.803892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a754c-6b17-4030-9f89-e7c3491cb15b service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:36.804169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a754c-6b17-4030-9f89-e7c3491cb15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:36.805828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-791a754c-6b17-4030-9f89-e7c3491cb15b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:36.922821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a754c-6b17-4030-9f89-e7c3491cb15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:36.924173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a754c-6b17-4030-9f89-e7c3491cb15b service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:36.944191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 718/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:39.305555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952376bb-df90-40a1-8d2b-6b9a9c21350b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:39.306094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952376bb-df90-40a1-8d2b-6b9a9c21350b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:39.306207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952376bb-df90-40a1-8d2b-6b9a9c21350b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:39.307591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:39.313103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a6073-d443-4012-836c-7aacd7019347 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:39.313558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a6073-d443-4012-836c-7aacd7019347 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:39.313692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a6073-d443-4012-836c-7aacd7019347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:39.380062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:39.382940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2e8a6073-d443-4012-836c-7aacd7019347 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:39.384662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2e8a6073-d443-4012-836c-7aacd7019347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:39.481192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:39.768799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952376bb-df90-40a1-8d2b-6b9a9c21350b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:39.770398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-952376bb-df90-40a1-8d2b-6b9a9c21350b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:39.884024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952376bb-df90-40a1-8d2b-6b9a9c21350b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:39.934733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:39.960873 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fba2e4-4504-46c0-a28e-5374aadbb996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:39.961521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fba2e4-4504-46c0-a28e-5374aadbb996 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:39.961815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fba2e4-4504-46c0-a28e-5374aadbb996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:39.962089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fba2e4-4504-46c0-a28e-5374aadbb996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:39.963724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-23fba2e4-4504-46c0-a28e-5374aadbb996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:40.076238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fba2e4-4504-46c0-a28e-5374aadbb996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:40.078439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fba2e4-4504-46c0-a28e-5374aadbb996 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:40.099337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 736/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:42.400112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74bd2-050f-4b74-ac1e-7a4a0b5709a3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:42.400788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74bd2-050f-4b74-ac1e-7a4a0b5709a3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:42.400788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74bd2-050f-4b74-ac1e-7a4a0b5709a3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:42.402040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:42.407979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6090-6869-4c6a-92fb-be260f692e26 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:42.408469 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6090-6869-4c6a-92fb-be260f692e26 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:42.408567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6090-6869-4c6a-92fb-be260f692e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:42.475714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-125a6090-6869-4c6a-92fb-be260f692e26 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:42.476938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-125a6090-6869-4c6a-92fb-be260f692e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:42.507115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:42.580294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:42.862727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74bd2-050f-4b74-ac1e-7a4a0b5709a3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:42.864308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-61b74bd2-050f-4b74-ac1e-7a4a0b5709a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:42.976477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74bd2-050f-4b74-ac1e-7a4a0b5709a3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:43.007243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:43.052646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b456471-9204-433d-94f7-7c5888bd91a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:43.053281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456471-9204-433d-94f7-7c5888bd91a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:43.053599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456471-9204-433d-94f7-7c5888bd91a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:43.053878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b456471-9204-433d-94f7-7c5888bd91a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:43.055574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2b456471-9204-433d-94f7-7c5888bd91a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:43.168645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b456471-9204-433d-94f7-7c5888bd91a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:43.170069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b456471-9204-433d-94f7-7c5888bd91a9 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:43.189647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 721/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:45.460833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1ead-eaf3-4b7d-a4a6-c94ffc9b46ef None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:45.461369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1ead-eaf3-4b7d-a4a6-c94ffc9b46ef None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:45.461369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1ead-eaf3-4b7d-a4a6-c94ffc9b46ef None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:45.462936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:45.469054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de36f9-736b-43b4-9ee1-33b2627daffc None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:45.469540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de36f9-736b-43b4-9ee1-33b2627daffc None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:45.469649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de36f9-736b-43b4-9ee1-33b2627daffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:45.542173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a4de36f9-736b-43b4-9ee1-33b2627daffc None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:45.543326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a4de36f9-736b-43b4-9ee1-33b2627daffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:45.553007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:45.653009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:45.939447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1ead-eaf3-4b7d-a4a6-c94ffc9b46ef None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:45.941556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8cbc1ead-eaf3-4b7d-a4a6-c94ffc9b46ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:46.049810 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1ead-eaf3-4b7d-a4a6-c94ffc9b46ef None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:46.109586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:46.130631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce5cc4-2008-4471-8305-71c01f5cc64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:46.131251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce5cc4-2008-4471-8305-71c01f5cc64c service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:46.131537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce5cc4-2008-4471-8305-71c01f5cc64c service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:46.131815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce5cc4-2008-4471-8305-71c01f5cc64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:46.133470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4bce5cc4-2008-4471-8305-71c01f5cc64c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:46.256450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce5cc4-2008-4471-8305-71c01f5cc64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:46.257842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce5cc4-2008-4471-8305-71c01f5cc64c service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:46.277999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 739/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:48.557654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7f01-fbfd-4726-a8ed-d15403dd49e0 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:48.558323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7f01-fbfd-4726-a8ed-d15403dd49e0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:48.558323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7f01-fbfd-4726-a8ed-d15403dd49e0 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:48.559593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:48.566179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8db9c-452e-4f48-8df1-24e585c9c9e3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:48.566957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8db9c-452e-4f48-8df1-24e585c9c9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:48.566957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8db9c-452e-4f48-8df1-24e585c9c9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:48.645061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dfd8db9c-452e-4f48-8df1-24e585c9c9e3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:48.646236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dfd8db9c-452e-4f48-8df1-24e585c9c9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:48.686294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:48.753984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:49.042509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d7f01-fbfd-4726-a8ed-d15403dd49e0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:49.044057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-434d7f01-fbfd-4726-a8ed-d15403dd49e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:49.151899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d7f01-fbfd-4726-a8ed-d15403dd49e0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:49.192957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:49.234513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab1fdd-71c6-4aaa-87da-e9e602786d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:49.235132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1fdd-71c6-4aaa-87da-e9e602786d49 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:49.235417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1fdd-71c6-4aaa-87da-e9e602786d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:49.235694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1fdd-71c6-4aaa-87da-e9e602786d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:49.237342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fcab1fdd-71c6-4aaa-87da-e9e602786d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:49.367469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab1fdd-71c6-4aaa-87da-e9e602786d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:49.368949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab1fdd-71c6-4aaa-87da-e9e602786d49 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:49.389811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 724/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:51.614769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f5017-c3ba-45bf-9f48-4a8fefd6d211 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:51.615316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f5017-c3ba-45bf-9f48-4a8fefd6d211 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:51.615316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f5017-c3ba-45bf-9f48-4a8fefd6d211 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:51.616681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:51.622518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677f34a-1ef1-4f02-b9b4-6ec8694647b1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:51.623006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677f34a-1ef1-4f02-b9b4-6ec8694647b1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:51.623152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677f34a-1ef1-4f02-b9b4-6ec8694647b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:51.698445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5677f34a-1ef1-4f02-b9b4-6ec8694647b1 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:51.699695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5677f34a-1ef1-4f02-b9b4-6ec8694647b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:51.719199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:51.808775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:52.093934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f5017-c3ba-45bf-9f48-4a8fefd6d211 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:52.095503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3d1f5017-c3ba-45bf-9f48-4a8fefd6d211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:52.207044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f5017-c3ba-45bf-9f48-4a8fefd6d211 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:52.277207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:52.284837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f873b0-7801-4c09-9ef3-387fde151b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:52.285498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f873b0-7801-4c09-9ef3-387fde151b54 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:52.285785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f873b0-7801-4c09-9ef3-387fde151b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:52.286067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f873b0-7801-4c09-9ef3-387fde151b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:52.287734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-89f873b0-7801-4c09-9ef3-387fde151b54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:52.413150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f873b0-7801-4c09-9ef3-387fde151b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:52.414547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f873b0-7801-4c09-9ef3-387fde151b54 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:52.434110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 742/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:24:54.767264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84b253-b0dd-4b5e-b807-90737b66b163 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:54.767844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84b253-b0dd-4b5e-b807-90737b66b163 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:54.767844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84b253-b0dd-4b5e-b807-90737b66b163 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:54.769224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:24:54.774826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd2f7-8145-48fb-bb95-b13048296156 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:54.775238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd2f7-8145-48fb-bb95-b13048296156 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:54.775385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfd2f7-8145-48fb-bb95-b13048296156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:54.844106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-44dfd2f7-8145-48fb-bb95-b13048296156 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:24:54.845364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-44dfd2f7-8145-48fb-bb95-b13048296156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:54.901228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:54.952928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:24:55.238914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84b253-b0dd-4b5e-b807-90737b66b163 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:24:55.240474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad84b253-b0dd-4b5e-b807-90737b66b163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:55.346628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84b253-b0dd-4b5e-b807-90737b66b163 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:24:55.396457 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:24:55.423544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cd6ff-4ead-494d-a658-b5ebaee305e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:24:55.424166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cd6ff-4ead-494d-a658-b5ebaee305e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:24:55.424452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cd6ff-4ead-494d-a658-b5ebaee305e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:24:55.424727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cd6ff-4ead-494d-a658-b5ebaee305e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:24:55.426386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-064cd6ff-4ead-494d-a658-b5ebaee305e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:24:55.539894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cd6ff-4ead-494d-a658-b5ebaee305e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:24:55.541262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cd6ff-4ead-494d-a658-b5ebaee305e4 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:24:55.559837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 727/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:25:04.950679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=22 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:05.557637 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=66 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:16.198856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df0546-9a3d-4cf6-b583-aa5923fab791 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:16.199422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df0546-9a3d-4cf6-b583-aa5923fab791 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:16.199422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df0546-9a3d-4cf6-b583-aa5923fab791 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:16.200902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 744/1471] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:25:16.207140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db858ec-1d22-499d-8888-f5328e157b41 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:16.207681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db858ec-1d22-499d-8888-f5328e157b41 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:16.207681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db858ec-1d22-499d-8888-f5328e157b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:16.275282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2db858ec-1d22-499d-8888-f5328e157b41 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:25:16.276508 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2db858ec-1d22-499d-8888-f5328e157b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:16.381174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 728/1472] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:25:16.695612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab331d7-6dc2-4234-ae52-ac80519b4b6f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:16.695893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab331d7-6dc2-4234-ae52-ac80519b4b6f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:16.696180 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab331d7-6dc2-4234-ae52-ac80519b4b6f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:16.697556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 745/1473] 10.222.0.58 () {54 vars in 962 bytes} [Mon Oct 3 13:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:25:16.702709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d920974-d0f7-473d-93cc-047dc0784889 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:16.702999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d920974-d0f7-473d-93cc-047dc0784889 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:16.703274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d920974-d0f7-473d-93cc-047dc0784889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:16.781408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9d920974-d0f7-473d-93cc-047dc0784889 None None] MFA Rules not processed for user `464190e0ccaf4f82942f518df1bf6c03`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:25:16.782674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9d920974-d0f7-473d-93cc-047dc0784889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:16.804837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:16.865289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 729/1474] 10.222.0.58 () {58 vars in 1059 bytes} [Mon Oct 3 13:25:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:25:16.873679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab331d7-6dc2-4234-ae52-ac80519b4b6f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:25:16.875322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aab331d7-6dc2-4234-ae52-ac80519b4b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:16.988517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab331d7-6dc2-4234-ae52-ac80519b4b6f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:25:17.063452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f14da-3a90-42dc-a4c9-c6c67d669008 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '464190e0ccaf4f82942f518df1bf6c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:25:17.064066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f14da-3a90-42dc-a4c9-c6c67d669008 service glance] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:17.064355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f14da-3a90-42dc-a4c9-c6c67d669008 service glance] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:17.064636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f14da-3a90-42dc-a4c9-c6c67d669008 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:17.066291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-863f14da-3a90-42dc-a4c9-c6c67d669008 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:17.121254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:17.179565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f14da-3a90-42dc-a4c9-c6c67d669008 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:25:17.181021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f14da-3a90-42dc-a4c9-c6c67d669008 service glance] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:25:17.199967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 746/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:25:19.499809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818072-7cf2-44e1-9a76-50393ba362e1 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:19.500451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818072-7cf2-44e1-9a76-50393ba362e1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:19.500451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818072-7cf2-44e1-9a76-50393ba362e1 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:19.501749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 730/1476] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:25:19.507735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfd209-697b-4419-9454-92a63dd2e8ab None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:19.508226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfd209-697b-4419-9454-92a63dd2e8ab None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:19.508331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfd209-697b-4419-9454-92a63dd2e8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:19.578014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-86cfd209-697b-4419-9454-92a63dd2e8ab None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:25:19.579201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-86cfd209-697b-4419-9454-92a63dd2e8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:19.679479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 747/1477] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:25:19.956166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab818072-7cf2-44e1-9a76-50393ba362e1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:25:19.957780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ab818072-7cf2-44e1-9a76-50393ba362e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:20.085333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab818072-7cf2-44e1-9a76-50393ba362e1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:25:20.135546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:20.182220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47320467-b607-4769-a4bc-750374ad6bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:25:20.182830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47320467-b607-4769-a4bc-750374ad6bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:20.183109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47320467-b607-4769-a4bc-750374ad6bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:20.183381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47320467-b607-4769-a4bc-750374ad6bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:20.184011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47320467-b607-4769-a4bc-750374ad6bc4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:25:20.186386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-47320467-b607-4769-a4bc-750374ad6bc4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:25:20.186386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Oct 03 13:25:20.186386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: Oct 03 13:25:20.187540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47320467-b607-4769-a4bc-750374ad6bc4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:25:20.207555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 731/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Mon Oct 3 13:25:19 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:25:22.432877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f99da1-6f2f-4e9f-9af4-c5f628192d09 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:22.433513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f99da1-6f2f-4e9f-9af4-c5f628192d09 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:22.433513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f99da1-6f2f-4e9f-9af4-c5f628192d09 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:22.434878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 748/1479] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:25:22.441156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752097-f94c-4ce3-b217-6098fc3cc8cb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:22.441675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752097-f94c-4ce3-b217-6098fc3cc8cb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:22.441773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752097-f94c-4ce3-b217-6098fc3cc8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:22.520814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-51752097-f94c-4ce3-b217-6098fc3cc8cb None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:25:22.523477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-51752097-f94c-4ce3-b217-6098fc3cc8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:22.646378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 732/1480] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:25:22.953062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f99da1-6f2f-4e9f-9af4-c5f628192d09 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:25:22.954754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-89f99da1-6f2f-4e9f-9af4-c5f628192d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:22.966758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:23.137599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f99da1-6f2f-4e9f-9af4-c5f628192d09 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:25:23.244574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:23.281532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e689ff-ef2b-4b72-9990-8d4268dab967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:25:23.282167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e689ff-ef2b-4b72-9990-8d4268dab967 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:23.282445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e689ff-ef2b-4b72-9990-8d4268dab967 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:23.282726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e689ff-ef2b-4b72-9990-8d4268dab967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:23.284405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-11e689ff-ef2b-4b72-9990-8d4268dab967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:23.401906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e689ff-ef2b-4b72-9990-8d4268dab967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:25:23.403527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e689ff-ef2b-4b72-9990-8d4268dab967 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:25:23.423962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 749/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:25:25.863399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0fc1-10c4-41e5-9557-da1324e42779 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:25.864099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0fc1-10c4-41e5-9557-da1324e42779 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:25.864099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac0fc1-10c4-41e5-9557-da1324e42779 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:25.865391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 733/1482] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:25:25.870927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e137-87cf-4694-ad76-3b7531a9519d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:25.871435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e137-87cf-4694-ad76-3b7531a9519d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:25.871548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e137-87cf-4694-ad76-3b7531a9519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:25.944875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f769e137-87cf-4694-ad76-3b7531a9519d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:25:25.946312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f769e137-87cf-4694-ad76-3b7531a9519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:25.997492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:26.051293 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 750/1483] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:25:26.344377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0fc1-10c4-41e5-9557-da1324e42779 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:25:26.345959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-21ac0fc1-10c4-41e5-9557-da1324e42779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:26.369958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:26.451133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac0fc1-10c4-41e5-9557-da1324e42779 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:25:26.530078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83098e-775c-40b3-a830-19f4a99867d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:25:26.530696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83098e-775c-40b3-a830-19f4a99867d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:26.530981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83098e-775c-40b3-a830-19f4a99867d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:26.531259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83098e-775c-40b3-a830-19f4a99867d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:26.532915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0d83098e-775c-40b3-a830-19f4a99867d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:26.618617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:26.657274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83098e-775c-40b3-a830-19f4a99867d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:25:26.658761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83098e-775c-40b3-a830-19f4a99867d3 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:25:26.678122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 734/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:25:28.982977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058fa04-c91a-443c-9d69-ce9481d2f022 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:28.983559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058fa04-c91a-443c-9d69-ce9481d2f022 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:28.983559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058fa04-c91a-443c-9d69-ce9481d2f022 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:28.984872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 751/1485] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:25:28.990524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8103cf3-c814-4964-a7fd-d8d6b9a34059 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:28.990978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8103cf3-c814-4964-a7fd-d8d6b9a34059 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:28.991084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8103cf3-c814-4964-a7fd-d8d6b9a34059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:29.062877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a8103cf3-c814-4964-a7fd-d8d6b9a34059 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:25:29.064100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a8103cf3-c814-4964-a7fd-d8d6b9a34059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:29.166808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 735/1486] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:25:29.459394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058fa04-c91a-443c-9d69-ce9481d2f022 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:25:29.460951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9058fa04-c91a-443c-9d69-ce9481d2f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:29.569611 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058fa04-c91a-443c-9d69-ce9481d2f022 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:25:29.643571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:29.645098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1989139-dd02-415a-b940-d82fce593e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:25:29.645812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1989139-dd02-415a-b940-d82fce593e89 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:29.646443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1989139-dd02-415a-b940-d82fce593e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:29.646746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1989139-dd02-415a-b940-d82fce593e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:29.649795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e1989139-dd02-415a-b940-d82fce593e89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:29.761062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1989139-dd02-415a-b940-d82fce593e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:25:29.762526 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1989139-dd02-415a-b940-d82fce593e89 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:25:29.781535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 752/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:25:32.136284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ef311-6e95-447d-8781-1f30b9b8b3ec None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:32.136851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ef311-6e95-447d-8781-1f30b9b8b3ec None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:32.136851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ef311-6e95-447d-8781-1f30b9b8b3ec None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:32.138864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 736/1488] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:25:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:25:32.144446 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c1a0-e20d-4c49-9a9f-42a2ced69881 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:32.144877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c1a0-e20d-4c49-9a9f-42a2ced69881 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:32.145043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4c1a0-e20d-4c49-9a9f-42a2ced69881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:32.213280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9fc4c1a0-e20d-4c49-9a9f-42a2ced69881 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:25:32.214547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9fc4c1a0-e20d-4c49-9a9f-42a2ced69881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:32.275189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:32.323231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 753/1489] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:25:32.623011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ef311-6e95-447d-8781-1f30b9b8b3ec None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:25:32.624600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c83ef311-6e95-447d-8781-1f30b9b8b3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:32.662274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:32.732364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ef311-6e95-447d-8781-1f30b9b8b3ec None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:25:32.808361 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2782d3a-1728-4f92-b2a2-b846cccc3b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:25:32.808985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782d3a-1728-4f92-b2a2-b846cccc3b7c service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:32.809267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782d3a-1728-4f92-b2a2-b846cccc3b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:32.809573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2782d3a-1728-4f92-b2a2-b846cccc3b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:32.812048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d2782d3a-1728-4f92-b2a2-b846cccc3b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:32.894881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:32.925379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2782d3a-1728-4f92-b2a2-b846cccc3b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:25:32.926854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2782d3a-1728-4f92-b2a2-b846cccc3b7c service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:25:32.945528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 737/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:25:35.396456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192b1b-a41c-434f-b02e-0bc680114ef5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:35.397055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192b1b-a41c-434f-b02e-0bc680114ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:35.397055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71192b1b-a41c-434f-b02e-0bc680114ef5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:35.398411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 754/1491] 10.222.0.58 () {54 vars in 927 bytes} [Mon Oct 3 13:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:25:35.403862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29cfa9-f32b-4c64-88ef-d6db6f032175 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:35.404327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29cfa9-f32b-4c64-88ef-d6db6f032175 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:35.404436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29cfa9-f32b-4c64-88ef-d6db6f032175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:35.471039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7c29cfa9-f32b-4c64-88ef-d6db6f032175 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:25:35.472242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7c29cfa9-f32b-4c64-88ef-d6db6f032175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:35.573888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 738/1492] 10.222.0.58 () {58 vars in 1024 bytes} [Mon Oct 3 13:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:25:35.866612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192b1b-a41c-434f-b02e-0bc680114ef5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:25:35.868150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-71192b1b-a41c-434f-b02e-0bc680114ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:35.976260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71192b1b-a41c-434f-b02e-0bc680114ef5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:25:36.052731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d300b2-14c7-4d2d-aef9-a17237184e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:25:36.053342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d300b2-14c7-4d2d-aef9-a17237184e2d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:25:36.053653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d300b2-14c7-4d2d-aef9-a17237184e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:25:36.053926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d300b2-14c7-4d2d-aef9-a17237184e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:25:36.055542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f0d300b2-14c7-4d2d-aef9-a17237184e2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:25:36.065130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:36.178547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d300b2-14c7-4d2d-aef9-a17237184e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:25:36.180041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d300b2-14c7-4d2d-aef9-a17237184e2d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:25:36.203543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 755/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:25:45.572204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=82 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:25:46.200738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=50 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:50.358211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cdd41-6cb8-453c-8a99-160e93a6e82c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:50.358826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cdd41-6cb8-453c-8a99-160e93a6e82c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:50.358826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cdd41-6cb8-453c-8a99-160e93a6e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:50.452343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4e5cdd41-6cb8-453c-8a99-160e93a6e82c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:26:50.453614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4e5cdd41-6cb8-453c-8a99-160e93a6e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:50.564592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 739/1494] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:26:50.572745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf06312-c8c9-4c53-a268-4d3f36c8a154 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:50.573382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf06312-c8c9-4c53-a268-4d3f36c8a154 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:50.573382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf06312-c8c9-4c53-a268-4d3f36c8a154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:50.655520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ebf06312-c8c9-4c53-a268-4d3f36c8a154 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:26:50.656685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ebf06312-c8c9-4c53-a268-4d3f36c8a154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:50.757092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 756/1495] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:26:50.762848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cdd41-6cb8-453c-8a99-160e93a6e82c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:50.764112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4e5cdd41-6cb8-453c-8a99-160e93a6e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:50.775682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cdd41-6cb8-453c-8a99-160e93a6e82c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:50.839962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:50.876614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:50.877234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:50.877543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:50.877817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:50.878465 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:26:50.879031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:50.880060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:26:50.880060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Oct 03 13:26:50.880060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: Oct 03 13:26:50.880959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:50.893573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 740/1496] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:26:50.902478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf06312-c8c9-4c53-a268-4d3f36c8a154 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:50.912429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf06312-c8c9-4c53-a268-4d3f36c8a154 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:50.978189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:51.016058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:51.016681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:51.016967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:51.017245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:51.017936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:51.019718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:26:51.019718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 03 13:26:51.019718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: Oct 03 13:26:51.020928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:51.082951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 757/1497] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:26:50 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:26:51.085976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:51.087553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:51.123246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f3dc5-08cf-4d5e-8871-caffecf84624 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:51.225593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:51.226301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:51.226510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:51.226790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:51.227463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:51.228514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:26:51.228514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Oct 03 13:26:51.228514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: Oct 03 13:26:51.229918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:51.277864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 741/1498] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:26:51 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:26:51.282109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:51.283702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:51.348919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0ff0-9c47-4a37-868b-a077fa96164f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:51.436800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:51.448308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:51.449253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:51.449652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:51.449948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:51.450651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:51.451754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:26:51.451754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 03 13:26:51.451754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: Oct 03 13:26:51.452694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:51.466039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 758/1499] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:26:51.470137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:51.471676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:51.484251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d079ca6-7508-41a1-8542-b8ea3bc0de41 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:51.512924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:51.587742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfa3d1-fc06-4a05-a637-e3435888b0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:51.588367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfa3d1-fc06-4a05-a637-e3435888b0ff admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:51.588649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfa3d1-fc06-4a05-a637-e3435888b0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:51.588931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfa3d1-fc06-4a05-a637-e3435888b0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:51.589935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfa3d1-fc06-4a05-a637-e3435888b0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:51.590982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-9bcfa3d1-fc06-4a05-a637-e3435888b0ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:26:51.590982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Oct 03 13:26:51.590982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: Oct 03 13:26:51.591878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfa3d1-fc06-4a05-a637-e3435888b0ff admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:51.605687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 742/1500] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:26:51.610446 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:51.612242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:51.648857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1deeca1-6344-4d99-beaf-98666d05239d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:51.765166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2250cf-6a52-4596-9939-31bb89334cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:51.765807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2250cf-6a52-4596-9939-31bb89334cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:51.766094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2250cf-6a52-4596-9939-31bb89334cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:51.766376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2250cf-6a52-4596-9939-31bb89334cd9 admin admin] PATH_INFO: `/v3/projects/ca19c7fe5e25441e857830c729f457af/users/6233c900f76649429acba613a1acf00e/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:51.794537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2250cf-6a52-4596-9939-31bb89334cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca19c7fe5e25441e857830c729f457af, user_id=6233c900f76649429acba613a1acf00e, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:51.795619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-4e2250cf-6a52-4596-9939-31bb89334cd9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:26:51.795619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 03 13:26:51.795619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: Oct 03 13:26:51.797660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2250cf-6a52-4596-9939-31bb89334cd9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:51.825979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 759/1501] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:26:51 2022] PUT /identity/v3/projects/ca19c7fe5e25441e857830c729f457af/users/6233c900f76649429acba613a1acf00e/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:26:51.838604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d274ee-e77c-4a47-9301-ba6b93503e91 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:51.839426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d274ee-e77c-4a47-9301-ba6b93503e91 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:51.839745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d274ee-e77c-4a47-9301-ba6b93503e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:51.900343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c5d274ee-e77c-4a47-9301-ba6b93503e91 None None] MFA Rules not processed for user `6233c900f76649429acba613a1acf00e`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:26:51.902185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c5d274ee-e77c-4a47-9301-ba6b93503e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:51.987436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 743/1502] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:26:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:26:52.005293 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436c46-a83b-40d3-870d-3685de4c556e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:52.005604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436c46-a83b-40d3-870d-3685de4c556e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:52.005889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436c46-a83b-40d3-870d-3685de4c556e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:52.007279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 760/1503] 10.222.0.58 () {54 vars in 963 bytes} [Mon Oct 3 13:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:26:52.014523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5dd1-c9aa-422c-bfa3-8491fece0ae1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:52.014815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5dd1-c9aa-422c-bfa3-8491fece0ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:52.015122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5dd1-c9aa-422c-bfa3-8491fece0ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:52.050952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:52.225227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1c7e5dd1-c9aa-422c-bfa3-8491fece0ae1 None None] MFA Rules not processed for user `8fa402682fd24228a77801edf1bb4d59`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:26:52.226523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1c7e5dd1-c9aa-422c-bfa3-8491fece0ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:52.305909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 744/1504] 10.222.0.58 () {58 vars in 1060 bytes} [Mon Oct 3 13:26:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:26:52.315770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d436c46-a83b-40d3-870d-3685de4c556e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:52.317317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6d436c46-a83b-40d3-870d-3685de4c556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:52.377765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:52.436156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d436c46-a83b-40d3-870d-3685de4c556e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:52.515259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:52.515874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:52.516157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:52.516441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:52.518097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:52.623088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:52.624450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:52.628184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:52.643498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 761/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:26:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:26:52.693362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178476-2724-4be4-b0a1-9db7592ef28e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:52.693846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178476-2724-4be4-b0a1-9db7592ef28e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:52.693974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a178476-2724-4be4-b0a1-9db7592ef28e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:52.695274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 745/1506] 10.222.0.58 () {52 vars in 830 bytes} [Mon Oct 3 13:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:26:52.699318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:52.700581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:52.802807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c1cd0-2634-4056-b492-68bc46728b20 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:52.913249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:52.913822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:52.914107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:52.914425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] PATH_INFO: `/v3/users/6233c900f76649429acba613a1acf00e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:52.929035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6233c900f76649429acba613a1acf00e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:52.929320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:26:52.929320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 03 13:26:52.929320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: Oct 03 13:26:52.933441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:52.941404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:53.007480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6233c900f76649429acba613a1acf00e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:53.032256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6233c900f76649429acba613a1acf00e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:53.032741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6233c900f76649429acba613a1acf00e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:53.042972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6233c900f76649429acba613a1acf00e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:53.058442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6233c900f76649429acba613a1acf00e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:53.065384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 762/1507] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:26:52 2022] DELETE /identity/v3/users/6233c900f76649429acba613a1acf00e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:26:53.069907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d1316-2b75-4356-a427-ef87ebca952f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:53.070199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d1316-2b75-4356-a427-ef87ebca952f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:53.070481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d1316-2b75-4356-a427-ef87ebca952f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:53.130703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ee0d1316-2b75-4356-a427-ef87ebca952f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:26:53.131946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ee0d1316-2b75-4356-a427-ef87ebca952f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:53.212569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:53.227166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 746/1508] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:26:53.237015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:53.238313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:53.343834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedae5a-0f1f-4305-aec4-cba0ab53c56e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:53.403027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:26:53.423901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332606a-4bb6-411e-9aa6-74c8e42f8811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:53.424515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332606a-4bb6-411e-9aa6-74c8e42f8811 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:53.424795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332606a-4bb6-411e-9aa6-74c8e42f8811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:53.425071 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332606a-4bb6-411e-9aa6-74c8e42f8811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:53.426759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1332606a-4bb6-411e-9aa6-74c8e42f8811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:53.537590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332606a-4bb6-411e-9aa6-74c8e42f8811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:53.538978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332606a-4bb6-411e-9aa6-74c8e42f8811 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:53.557342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 763/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:26:53.971124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d1316-2b75-4356-a427-ef87ebca952f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:26:53.972355 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ee0d1316-2b75-4356-a427-ef87ebca952f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:26:53.983991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d1316-2b75-4356-a427-ef87ebca952f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:26:54.085546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:26:54.086101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:26:54.086386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:26:54.086671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] PATH_INFO: `/v3/projects/ca19c7fe5e25441e857830c729f457af` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:26:54.097690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca19c7fe5e25441e857830c729f457af)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:26:54.098734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:26:54.098734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Oct 03 13:26:54.098734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: Oct 03 13:26:54.099985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:26:54.158998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca19c7fe5e25441e857830c729f457af', 'request_id': 'req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:54.172513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca19c7fe5e25441e857830c729f457af', 'request_id': 'req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:54.178739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca19c7fe5e25441e857830c729f457af', 'request_id': 'req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:54.206016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] The token cache is being invalidate because project ca19c7fe5e25441e857830c729f457af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:26:54.206311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:54.206745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b014e60b-d8a3-4c72-b031-4c0666b9bc3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:26:54.208152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 747/1510] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:26:53 2022] DELETE /identity/v3/projects/ca19c7fe5e25441e857830c729f457af => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:27:03.555041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=62 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:27:04.203885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:28:27.095846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd2db2-9a89-4067-bd31-dfd0c22f1ad0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:27.096494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd2db2-9a89-4067-bd31-dfd0c22f1ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:27.096494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd2db2-9a89-4067-bd31-dfd0c22f1ad0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:27.097816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:27.102708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ac296-33e1-405c-8cc1-1df18f5fb6d0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:27.103186 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ac296-33e1-405c-8cc1-1df18f5fb6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:27.103290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ac296-33e1-405c-8cc1-1df18f5fb6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:27.185601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5f5ac296-33e1-405c-8cc1-1df18f5fb6d0 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:27.186818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5f5ac296-33e1-405c-8cc1-1df18f5fb6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:27.289786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:28:27.444000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd2db2-9a89-4067-bd31-dfd0c22f1ad0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:27.446116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d5fd2db2-9a89-4067-bd31-dfd0c22f1ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:27.568355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd2db2-9a89-4067-bd31-dfd0c22f1ad0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:27.574176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23709e9e-cd6a-419a-8663-6a09e202a268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:27.574790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23709e9e-cd6a-419a-8663-6a09e202a268 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:27.575101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23709e9e-cd6a-419a-8663-6a09e202a268 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:27.576052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23709e9e-cd6a-419a-8663-6a09e202a268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:27.584819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23709e9e-cd6a-419a-8663-6a09e202a268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:27.586442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23709e9e-cd6a-419a-8663-6a09e202a268 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:27.596706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 765/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:28:28.709936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671f7bc-b905-4ab6-9e1f-4d58b1ba9491 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:28.710533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671f7bc-b905-4ab6-9e1f-4d58b1ba9491 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:28.710533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671f7bc-b905-4ab6-9e1f-4d58b1ba9491 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:28.711953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:28.720067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a467d-742a-4846-9299-f3795b7f953d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:28.720593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a467d-742a-4846-9299-f3795b7f953d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:28.720593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a467d-742a-4846-9299-f3795b7f953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:28.740395 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d01a467d-742a-4846-9299-f3795b7f953d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:28.741624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d01a467d-742a-4846-9299-f3795b7f953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:28.749509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:28:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:28:28.903624 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671f7bc-b905-4ab6-9e1f-4d58b1ba9491 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:28.917990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671f7bc-b905-4ab6-9e1f-4d58b1ba9491 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:28.923732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ab0f6-45eb-4fab-8311-31b4d896ebad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:28.924335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab0f6-45eb-4fab-8311-31b4d896ebad service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:28.924622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab0f6-45eb-4fab-8311-31b4d896ebad service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:28.924901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab0f6-45eb-4fab-8311-31b4d896ebad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:28.932928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ab0f6-45eb-4fab-8311-31b4d896ebad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:28.935077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ab0f6-45eb-4fab-8311-31b4d896ebad service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:28.944573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 750/1516] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:28:31.520360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3afe0b-4668-47b9-9eef-a2364258467d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:31.521639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3afe0b-4668-47b9-9eef-a2364258467d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:31.521926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3afe0b-4668-47b9-9eef-a2364258467d None None] PATH_INFO: `/v3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:31.523506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 767/1517] 10.222.0.65 () {54 vars in 925 bytes} [Mon Oct 3 13:28:31 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:28:31.532828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5985c3-a0b3-4aa9-b6dc-72bb0c7cfab3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:31.533375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5985c3-a0b3-4aa9-b6dc-72bb0c7cfab3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:31.533512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5985c3-a0b3-4aa9-b6dc-72bb0c7cfab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:31.593897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1f5985c3-a0b3-4aa9-b6dc-72bb0c7cfab3 None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:31.595129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1f5985c3-a0b3-4aa9-b6dc-72bb0c7cfab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:31.612514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:28:31.672494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 751/1518] 10.222.0.65 () {58 vars in 1015 bytes} [Mon Oct 3 13:28:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:28:35.861668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d801f-7050-41d1-8f59-7b213589740f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:35.862213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d801f-7050-41d1-8f59-7b213589740f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:35.862213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d801f-7050-41d1-8f59-7b213589740f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:35.863557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 768/1519] 10.222.0.58 () {54 vars in 961 bytes} [Mon Oct 3 13:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:35.868688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106cf3-84b1-4f22-a7ca-5dffbfec7ef3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:35.869156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106cf3-84b1-4f22-a7ca-5dffbfec7ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:35.869337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106cf3-84b1-4f22-a7ca-5dffbfec7ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:35.890688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-52106cf3-84b1-4f22-a7ca-5dffbfec7ef3 None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:35.891924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-52106cf3-84b1-4f22-a7ca-5dffbfec7ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:35.900474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 752/1520] 10.222.0.58 () {58 vars in 1058 bytes} [Mon Oct 3 13:28:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:28:35.908363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d801f-7050-41d1-8f59-7b213589740f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:35.923010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d801f-7050-41d1-8f59-7b213589740f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:35.928212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4c809-d263-46e0-99d8-3ebd445f8349 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:35.928817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4c809-d263-46e0-99d8-3ebd445f8349 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:35.929099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4c809-d263-46e0-99d8-3ebd445f8349 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:35.929373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4c809-d263-46e0-99d8-3ebd445f8349 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:35.938843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4c809-d263-46e0-99d8-3ebd445f8349 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:35.940439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4c809-d263-46e0-99d8-3ebd445f8349 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:35.950400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 769/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:28:40.462269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b405e-ed05-4fbf-affd-23b14f07b2fb None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:40.462834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b405e-ed05-4fbf-affd-23b14f07b2fb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:40.462834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b405e-ed05-4fbf-affd-23b14f07b2fb None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:40.464182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:40.471393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0108d0c-2543-44c1-ba74-f35e19e00ac2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:40.471891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0108d0c-2543-44c1-ba74-f35e19e00ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:40.471988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0108d0c-2543-44c1-ba74-f35e19e00ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:40.491718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e0108d0c-2543-44c1-ba74-f35e19e00ac2 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:40.492945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e0108d0c-2543-44c1-ba74-f35e19e00ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:40.501090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:28:40.656943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b405e-ed05-4fbf-affd-23b14f07b2fb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:40.671819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b405e-ed05-4fbf-affd-23b14f07b2fb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:40.677232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e1c1a-eefc-4b4e-a96e-92650a48a75f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:40.677872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e1c1a-eefc-4b4e-a96e-92650a48a75f service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:40.678156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e1c1a-eefc-4b4e-a96e-92650a48a75f service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:40.678432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e1c1a-eefc-4b4e-a96e-92650a48a75f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:40.687194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e1c1a-eefc-4b4e-a96e-92650a48a75f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:40.688857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087e1c1a-eefc-4b4e-a96e-92650a48a75f service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:40.698862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 754/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:28:41.793569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb87392-907d-4b39-8cba-542545f7f0e2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:41.794230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb87392-907d-4b39-8cba-542545f7f0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:41.794230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb87392-907d-4b39-8cba-542545f7f0e2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:41.795495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 3 13:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:41.801917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24605a64-8210-4e62-857a-c65b5128c3da None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:41.802404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24605a64-8210-4e62-857a-c65b5128c3da None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:41.802404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24605a64-8210-4e62-857a-c65b5128c3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:41.822147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-24605a64-8210-4e62-857a-c65b5128c3da None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:41.823320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-24605a64-8210-4e62-857a-c65b5128c3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:41.832029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 3 13:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:28:41.987895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb87392-907d-4b39-8cba-542545f7f0e2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:42.003288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb87392-907d-4b39-8cba-542545f7f0e2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:42.009073 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c51f-49be-4088-876a-a68a2c7c3640 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:42.009706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c51f-49be-4088-876a-a68a2c7c3640 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:42.009988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c51f-49be-4088-876a-a68a2c7c3640 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:42.010266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c51f-49be-4088-876a-a68a2c7c3640 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:42.018599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7c51f-49be-4088-876a-a68a2c7c3640 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:42.020183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7c51f-49be-4088-876a-a68a2c7c3640 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:42.029606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 772/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:28:46.195198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49184dab-6e81-4981-9855-0b21b003ef94 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:46.195755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49184dab-6e81-4981-9855-0b21b003ef94 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:46.195874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49184dab-6e81-4981-9855-0b21b003ef94 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:46.197282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:46.203115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdea43a-ab56-4fe0-8085-7fc5e1276938 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:46.203599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdea43a-ab56-4fe0-8085-7fc5e1276938 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:46.203706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdea43a-ab56-4fe0-8085-7fc5e1276938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:46.224754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9cdea43a-ab56-4fe0-8085-7fc5e1276938 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:46.226006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9cdea43a-ab56-4fe0-8085-7fc5e1276938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:46.234636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:28:46.523439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49184dab-6e81-4981-9855-0b21b003ef94 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:46.525569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-49184dab-6e81-4981-9855-0b21b003ef94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:46.614619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49184dab-6e81-4981-9855-0b21b003ef94 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:46.618933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706769f-7a62-48b4-b627-2261d0e3c238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:46.619556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706769f-7a62-48b4-b627-2261d0e3c238 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:46.619837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706769f-7a62-48b4-b627-2261d0e3c238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:46.620115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706769f-7a62-48b4-b627-2261d0e3c238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:46.629967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706769f-7a62-48b4-b627-2261d0e3c238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:46.631415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706769f-7a62-48b4-b627-2261d0e3c238 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:46.641376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 757/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:28:48.838341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84fd0-195d-4209-8d64-5c2c03ee95c9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:48.838923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84fd0-195d-4209-8d64-5c2c03ee95c9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:48.838923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84fd0-195d-4209-8d64-5c2c03ee95c9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:48.844409 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:48.846537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f9486-25b5-484a-872d-01b9d0559561 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:48.847354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f9486-25b5-484a-872d-01b9d0559561 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:48.847354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f9486-25b5-484a-872d-01b9d0559561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:48.867737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6a7f9486-25b5-484a-872d-01b9d0559561 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:48.868950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6a7f9486-25b5-484a-872d-01b9d0559561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:48.876880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:28:49.166440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84fd0-195d-4209-8d64-5c2c03ee95c9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:49.178332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84fd0-195d-4209-8d64-5c2c03ee95c9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:49.181943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf31770-84c8-49d3-b0e5-8ebc0396d927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:49.182550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31770-84c8-49d3-b0e5-8ebc0396d927 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:49.182835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31770-84c8-49d3-b0e5-8ebc0396d927 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:49.183116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31770-84c8-49d3-b0e5-8ebc0396d927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:49.193252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf31770-84c8-49d3-b0e5-8ebc0396d927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:49.194750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf31770-84c8-49d3-b0e5-8ebc0396d927 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:49.205424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 775/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:28:53.262171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9fec1-18f9-4e9e-b160-a97945837525 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:53.262722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9fec1-18f9-4e9e-b160-a97945837525 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:53.262722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9fec1-18f9-4e9e-b160-a97945837525 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:53.264245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:53.269752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e557-caab-48f7-b0f1-007307596548 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:53.270190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e557-caab-48f7-b0f1-007307596548 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:53.270312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25e557-caab-48f7-b0f1-007307596548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:53.290853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8d25e557-caab-48f7-b0f1-007307596548 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:53.292085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8d25e557-caab-48f7-b0f1-007307596548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:53.300751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:28:53.606035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9fec1-18f9-4e9e-b160-a97945837525 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:53.608382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3ae9fec1-18f9-4e9e-b160-a97945837525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:53.681660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:28:53.695373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9fec1-18f9-4e9e-b160-a97945837525 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:53.698919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ba6ce-91fd-4f8e-bebc-41ad17393af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:53.699530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ba6ce-91fd-4f8e-bebc-41ad17393af6 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:53.699806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ba6ce-91fd-4f8e-bebc-41ad17393af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:53.700078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ba6ce-91fd-4f8e-bebc-41ad17393af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:53.709933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ba6ce-91fd-4f8e-bebc-41ad17393af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:53.711366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ba6ce-91fd-4f8e-bebc-41ad17393af6 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:53.721702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 760/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:28:55.948408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e55ab-83c1-4f51-b6e3-666001fb2baf None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:55.949034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e55ab-83c1-4f51-b6e3-666001fb2baf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:55.949034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e55ab-83c1-4f51-b6e3-666001fb2baf None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:55.950358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 3 13:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:28:55.955770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09d37e-e39b-4eb4-8a54-a68e4b68cc87 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:55.956256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09d37e-e39b-4eb4-8a54-a68e4b68cc87 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:55.956362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e09d37e-e39b-4eb4-8a54-a68e4b68cc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:55.977716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6e09d37e-e39b-4eb4-8a54-a68e4b68cc87 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:28:55.978944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6e09d37e-e39b-4eb4-8a54-a68e4b68cc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:55.987351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 3 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:28:56.296468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e55ab-83c1-4f51-b6e3-666001fb2baf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:28:56.298214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3c8e55ab-83c1-4f51-b6e3-666001fb2baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:28:56.306023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:28:56.312549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e55ab-83c1-4f51-b6e3-666001fb2baf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:28:56.316500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d82c8-3d46-42a3-9db5-21d814790625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:28:56.317168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d82c8-3d46-42a3-9db5-21d814790625 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:28:56.317468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d82c8-3d46-42a3-9db5-21d814790625 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:28:56.317748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d82c8-3d46-42a3-9db5-21d814790625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:28:56.328023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d82c8-3d46-42a3-9db5-21d814790625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:28:56.329561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d82c8-3d46-42a3-9db5-21d814790625 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:28:56.340670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 778/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:29:05.972606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=11 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:29:06.335683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:25.804672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eae3b-cc86-41b5-8556-b6dbec60c162 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:25.805339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eae3b-cc86-41b5-8556-b6dbec60c162 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:25.805339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eae3b-cc86-41b5-8556-b6dbec60c162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:25.836423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cc3eae3b-cc86-41b5-8556-b6dbec60c162 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:25.837704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cc3eae3b-cc86-41b5-8556-b6dbec60c162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:25.853839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 762/1540] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:25.862738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a203b-decb-423c-8e8d-546e146e7cc7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:25.863778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a203b-decb-423c-8e8d-546e146e7cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:25.864275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a203b-decb-423c-8e8d-546e146e7cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:25.896042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-301a203b-decb-423c-8e8d-546e146e7cc7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:25.896042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-301a203b-decb-423c-8e8d-546e146e7cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:25.912277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 779/1541] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:25.918578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eae3b-cc86-41b5-8556-b6dbec60c162 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:25.942472 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eae3b-cc86-41b5-8556-b6dbec60c162 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:25.954137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880c759-cd3a-4912-b343-1b3faec3338f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:25.954775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c759-cd3a-4912-b343-1b3faec3338f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:25.955060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c759-cd3a-4912-b343-1b3faec3338f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:25.955329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880c759-cd3a-4912-b343-1b3faec3338f admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:25.955985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880c759-cd3a-4912-b343-1b3faec3338f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:25.956286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880c759-cd3a-4912-b343-1b3faec3338f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:25.957927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880c759-cd3a-4912-b343-1b3faec3338f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:25.983176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 763/1542] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:25.991519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a203b-decb-423c-8e8d-546e146e7cc7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:25.999680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062d230-4c8e-4a0c-a9e9-82feb2b8420d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.000773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062d230-4c8e-4a0c-a9e9-82feb2b8420d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.000773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062d230-4c8e-4a0c-a9e9-82feb2b8420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.007924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a203b-decb-423c-8e8d-546e146e7cc7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.017558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5a565-41b4-48a1-a8dd-5cf6521b090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.017558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5a565-41b4-48a1-a8dd-5cf6521b090f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.017558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5a565-41b4-48a1-a8dd-5cf6521b090f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.017558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5a565-41b4-48a1-a8dd-5cf6521b090f admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.017991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5a565-41b4-48a1-a8dd-5cf6521b090f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.018323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5a565-41b4-48a1-a8dd-5cf6521b090f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.050490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a062d230-4c8e-4a0c-a9e9-82feb2b8420d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.051732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a062d230-4c8e-4a0c-a9e9-82feb2b8420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.065249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 764/1543] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.071703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feee636-3bbd-4ede-bf28-f4255bf8877d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.072071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feee636-3bbd-4ede-bf28-f4255bf8877d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.072264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feee636-3bbd-4ede-bf28-f4255bf8877d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.100920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 780/1544] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:25 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:26.107129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1efb3-7210-44cc-afa2-fad9fbd8c578 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.107488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1efb3-7210-44cc-afa2-fad9fbd8c578 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.107752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1efb3-7210-44cc-afa2-fad9fbd8c578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.108488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7feee636-3bbd-4ede-bf28-f4255bf8877d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.110299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7feee636-3bbd-4ede-bf28-f4255bf8877d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.152590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6ea1efb3-7210-44cc-afa2-fad9fbd8c578 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.153927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6ea1efb3-7210-44cc-afa2-fad9fbd8c578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.241557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 765/1545] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.241557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc85d-92e4-4312-95f4-76f3d214f900 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.241557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc85d-92e4-4312-95f4-76f3d214f900 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.241557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc85d-92e4-4312-95f4-76f3d214f900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.251816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 781/1546] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.251816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1efb3-7210-44cc-afa2-fad9fbd8c578 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.261617 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1efb3-7210-44cc-afa2-fad9fbd8c578 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.266083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b6ecc85d-92e4-4312-95f4-76f3d214f900 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.267578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b6ecc85d-92e4-4312-95f4-76f3d214f900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.268511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f241b5-5ead-484a-92f0-90f0fbad3041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.269216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f241b5-5ead-484a-92f0-90f0fbad3041 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.269551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f241b5-5ead-484a-92f0-90f0fbad3041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.269897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f241b5-5ead-484a-92f0-90f0fbad3041 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.270474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f241b5-5ead-484a-92f0-90f0fbad3041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.271504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-71f241b5-5ead-484a-92f0-90f0fbad3041 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:30:26.271504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 03 13:30:26.271504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: Oct 03 13:30:26.272791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f241b5-5ead-484a-92f0-90f0fbad3041 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.280259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 766/1547] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.280259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee0a6-ded3-4168-809a-453633b85413 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.280259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee0a6-ded3-4168-809a-453633b85413 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.280259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee0a6-ded3-4168-809a-453633b85413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.299739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 782/1548] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/users => generated 288 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:26.303999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8cd54-b9a7-4a43-8c70-65e5cc88c36b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.303999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8cd54-b9a7-4a43-8c70-65e5cc88c36b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.303999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8cd54-b9a7-4a43-8c70-65e5cc88c36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.304380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c9aee0a6-ded3-4168-809a-453633b85413 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.304380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c9aee0a6-ded3-4168-809a-453633b85413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.314250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 767/1549] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.315647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a82287-5d65-4277-b3f6-500fdeaacccb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.315927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a82287-5d65-4277-b3f6-500fdeaacccb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.316115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a82287-5d65-4277-b3f6-500fdeaacccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.328378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-74c8cd54-b9a7-4a43-8c70-65e5cc88c36b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.330152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-74c8cd54-b9a7-4a43-8c70-65e5cc88c36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.341657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 783/1550] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.343116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbec053-2b16-41ac-9c78-e9951cbe5b36 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.343589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbec053-2b16-41ac-9c78-e9951cbe5b36 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.344505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbec053-2b16-41ac-9c78-e9951cbe5b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.349411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a3a82287-5d65-4277-b3f6-500fdeaacccb None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.350841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a3a82287-5d65-4277-b3f6-500fdeaacccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.359086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 768/1551] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.360963 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6e041-2e11-4c2c-9b85-e260a434050d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.361258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6e041-2e11-4c2c-9b85-e260a434050d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.361588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6e041-2e11-4c2c-9b85-e260a434050d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.372449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-edbec053-2b16-41ac-9c78-e9951cbe5b36 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.373712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-edbec053-2b16-41ac-9c78-e9951cbe5b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.382693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3ba6e041-2e11-4c2c-9b85-e260a434050d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.383309 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 784/1552] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.383940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbec053-2b16-41ac-9c78-e9951cbe5b36 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.384043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3ba6e041-2e11-4c2c-9b85-e260a434050d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.392311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 769/1553] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.393682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7055f2-aa8d-4092-a047-4840d04a8f9f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.393972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7055f2-aa8d-4092-a047-4840d04a8f9f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.394247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbec053-2b16-41ac-9c78-e9951cbe5b36 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.394364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7055f2-aa8d-4092-a047-4840d04a8f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.400779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.401416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.401735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.402054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.402723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:26.403040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.404146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:30:26.404146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 03 13:30:26.404146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: Oct 03 13:30:26.405014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.416890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0b7055f2-aa8d-4092-a047-4840d04a8f9f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.418109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0b7055f2-aa8d-4092-a047-4840d04a8f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.420138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 785/1554] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:26.420956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.430562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 770/1555] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.432157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9bd6f-7ac9-433f-a5b7-d8adafdcc1ad None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.432157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9bd6f-7ac9-433f-a5b7-d8adafdcc1ad None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.432428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9bd6f-7ac9-433f-a5b7-d8adafdcc1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.438446 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8063d4-c7a4-47d4-b901-7449b104eb10 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.444344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c261e-3221-4022-9597-2c00c1301f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.444973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c261e-3221-4022-9597-2c00c1301f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.445283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c261e-3221-4022-9597-2c00c1301f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.445606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c261e-3221-4022-9597-2c00c1301f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.446269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c261e-3221-4022-9597-2c00c1301f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.448007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c261e-3221-4022-9597-2c00c1301f05 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.454989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c7c9bd6f-7ac9-433f-a5b7-d8adafdcc1ad None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.456274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c7c9bd6f-7ac9-433f-a5b7-d8adafdcc1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.459438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 786/1556] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:26.460547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b7a8a-dd8a-457e-b6e4-2903508fbfa8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.460833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b7a8a-dd8a-457e-b6e4-2903508fbfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.461111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b7a8a-dd8a-457e-b6e4-2903508fbfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.467038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 771/1557] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.469190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6c31-f1bf-4603-999c-3ee18ae6c3fa None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.469504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6c31-f1bf-4603-999c-3ee18ae6c3fa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.469941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6c31-f1bf-4603-999c-3ee18ae6c3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.485981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-267b7a8a-dd8a-457e-b6e4-2903508fbfa8 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.487230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-267b7a8a-dd8a-457e-b6e4-2903508fbfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.495349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 787/1558] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.496148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7a8a6c31-f1bf-4603-999c-3ee18ae6c3fa None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.496270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b7a8a-dd8a-457e-b6e4-2903508fbfa8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.497337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7a8a6c31-f1bf-4603-999c-3ee18ae6c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.506421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 772/1559] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.506623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b7a8a-dd8a-457e-b6e4-2903508fbfa8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.507232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a6c31-f1bf-4603-999c-3ee18ae6c3fa None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.511860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.512503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.512787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.513064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.513863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:26.514359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.515753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.517005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a6c31-f1bf-4603-999c-3ee18ae6c3fa None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.523300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d447a-c092-45b2-aa48-7c48af71f577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.523999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d447a-c092-45b2-aa48-7c48af71f577 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.524286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d447a-c092-45b2-aa48-7c48af71f577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.524694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d447a-c092-45b2-aa48-7c48af71f577 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.525261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d447a-c092-45b2-aa48-7c48af71f577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:26.525609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d447a-c092-45b2-aa48-7c48af71f577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.527496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d447a-c092-45b2-aa48-7c48af71f577 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.528362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 788/1560] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:26.529427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.541697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb1cb5-07fa-4098-883c-e38563ad97a3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.551448 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 773/1561] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:26.551448 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b28a5-6cae-40bd-80f6-4568aa337d35 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.551448 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b28a5-6cae-40bd-80f6-4568aa337d35 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.551448 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b28a5-6cae-40bd-80f6-4568aa337d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.553937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558190d-25b2-4872-a750-d8c4e04112c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.554546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558190d-25b2-4872-a750-d8c4e04112c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.554954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558190d-25b2-4872-a750-d8c4e04112c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.555178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558190d-25b2-4872-a750-d8c4e04112c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.556126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558190d-25b2-4872-a750-d8c4e04112c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.557986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558190d-25b2-4872-a750-d8c4e04112c5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.569732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3f3b28a5-6cae-40bd-80f6-4568aa337d35 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:26.570868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3f3b28a5-6cae-40bd-80f6-4568aa337d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:26.580075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 774/1562] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:26.580754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b28a5-6cae-40bd-80f6-4568aa337d35 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.592044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 789/1563] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:26.592044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558190d-25b2-4872-a750-d8c4e04112c5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.592643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b28a5-6cae-40bd-80f6-4568aa337d35 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.629858 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:26.661330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f83ff-cdcc-4860-8ee6-7bce4c3a7bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.661990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f83ff-cdcc-4860-8ee6-7bce4c3a7bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.662275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f83ff-cdcc-4860-8ee6-7bce4c3a7bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.662551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f83ff-cdcc-4860-8ee6-7bce4c3a7bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.663208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f83ff-cdcc-4860-8ee6-7bce4c3a7bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.664740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558190d-25b2-4872-a750-d8c4e04112c5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.664925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f83ff-cdcc-4860-8ee6-7bce4c3a7bb3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.670166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.670808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.671093 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.671376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.672086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:26.672470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.674151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.676605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 775/1564] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:26.680614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f83ff-cdcc-4860-8ee6-7bce4c3a7bb3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.686707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:26.686870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 790/1565] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:26.688328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.692277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f83ff-cdcc-4860-8ee6-7bce4c3a7bb3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.702295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60dbb2-2f6d-4b2d-a083-9bf6d2d02b5c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.702511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.703815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.704331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.704539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.705203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:26.705552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.707191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.708346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.709004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.709286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.709603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.710264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:26.710575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.712181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.720527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 776/1566] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:26.721369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.729693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 791/1567] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:26.730968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.734993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5249ef-525c-480d-b9cb-2b236d775dbf admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.740049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.740824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.741108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.741380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.742058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.742837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de3425-4bf0-4de7-ab49-8869c67cf3e5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.743113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:30:26.743113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Oct 03 13:30:26.743113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: Oct 03 13:30:26.744311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.747523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d025e-3a1d-442a-a955-85110ce2b9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.748123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d025e-3a1d-442a-a955-85110ce2b9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.748541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d025e-3a1d-442a-a955-85110ce2b9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.748828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d025e-3a1d-442a-a955-85110ce2b9e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.749498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d025e-3a1d-442a-a955-85110ce2b9e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.752366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d025e-3a1d-442a-a955-85110ce2b9e1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.778949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 777/1568] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:26.779813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.784767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 792/1569] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:26.785647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d025e-3a1d-442a-a955-85110ce2b9e1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.848276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41b021-be21-4ed2-87a4-df5d5555e5f8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.853378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.854174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.854296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d025e-3a1d-442a-a955-85110ce2b9e1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.854466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.854741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.855390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:26.855693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.857346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.859006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38821cb-bd0e-41e8-b590-28b5dbf7c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.860426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38821cb-bd0e-41e8-b590-28b5dbf7c4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.860941 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38821cb-bd0e-41e8-b590-28b5dbf7c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.861238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38821cb-bd0e-41e8-b590-28b5dbf7c4a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.861922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38821cb-bd0e-41e8-b590-28b5dbf7c4a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.863867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38821cb-bd0e-41e8-b590-28b5dbf7c4a7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.870867 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 778/1570] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:26.871678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.882836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70381d-9873-48a4-ad6d-56e8acb9e1de admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.889087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.889877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.890181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.891150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] PATH_INFO: `/v3/projects/3d84507eb30f4a7fabd41f8962a98e06/users/82edf464a87948c9a806b5986b7e014b/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.900445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 793/1571] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:26.900445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38821cb-bd0e-41e8-b590-28b5dbf7c4a7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.907588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d84507eb30f4a7fabd41f8962a98e06, user_id=82edf464a87948c9a806b5986b7e014b, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.908657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:30:26.908657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Oct 03 13:30:26.908657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: Oct 03 13:30:26.910705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.911347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38821cb-bd0e-41e8-b590-28b5dbf7c4a7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:26.917386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cbc99-1d32-426d-8305-5efca5a7f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:26.918068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cbc99-1d32-426d-8305-5efca5a7f029 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:26.918363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cbc99-1d32-426d-8305-5efca5a7f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:26.918677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977cbc99-1d32-426d-8305-5efca5a7f029 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:26.919560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cbc99-1d32-426d-8305-5efca5a7f029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:26.921527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977cbc99-1d32-426d-8305-5efca5a7f029 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:26.926200 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 779/1572] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:26 2022] PUT /identity/v3/projects/3d84507eb30f4a7fabd41f8962a98e06/users/82edf464a87948c9a806b5986b7e014b/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:26.926948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:26.962669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 794/1573] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:26.962669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cbc99-1d32-426d-8305-5efca5a7f029 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.011964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04b4ea-4172-47f9-b287-0dd4f9b58633 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.021058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:27.034847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977cbc99-1d32-426d-8305-5efca5a7f029 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.039901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf08f-dc86-455d-bf87-a312c5a56aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.040885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf08f-dc86-455d-bf87-a312c5a56aea admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.041335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf08f-dc86-455d-bf87-a312c5a56aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.042037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf08f-dc86-455d-bf87-a312c5a56aea admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.042761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bf08f-dc86-455d-bf87-a312c5a56aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.045829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bf08f-dc86-455d-bf87-a312c5a56aea admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.062721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:27.077402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 795/1574] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.079100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e8363-d443-4de4-8e9d-55b851b60905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.079997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8363-d443-4de4-8e9d-55b851b60905 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.080277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8363-d443-4de4-8e9d-55b851b60905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.080546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e8363-d443-4de4-8e9d-55b851b60905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.081196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e8363-d443-4de4-8e9d-55b851b60905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.082112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf08f-dc86-455d-bf87-a312c5a56aea admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.083172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e8363-d443-4de4-8e9d-55b851b60905 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.115329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 780/1575] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:26 2022] POST /identity/v3/projects => generated 393 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.120613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e8363-d443-4de4-8e9d-55b851b60905 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.162365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf08f-dc86-455d-bf87-a312c5a56aea admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.204026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e8363-d443-4de4-8e9d-55b851b60905 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.210451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaaeff-bd54-43d1-ace3-b0ad07e82c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.211157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaaeff-bd54-43d1-ace3-b0ad07e82c20 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.211510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaaeff-bd54-43d1-ace3-b0ad07e82c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.211851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaaeff-bd54-43d1-ace3-b0ad07e82c20 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.212811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aaaeff-bd54-43d1-ace3-b0ad07e82c20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.215521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aaaeff-bd54-43d1-ace3-b0ad07e82c20 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.243351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 781/1576] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/users => generated 302 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.243351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaaeff-bd54-43d1-ace3-b0ad07e82c20 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.243663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc3e87-b5e5-40ac-9528-e87aad471d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.243663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc3e87-b5e5-40ac-9528-e87aad471d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.243663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc3e87-b5e5-40ac-9528-e87aad471d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.243663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc3e87-b5e5-40ac-9528-e87aad471d89 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.244044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc3e87-b5e5-40ac-9528-e87aad471d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.245365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc3e87-b5e5-40ac-9528-e87aad471d89 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.251948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaaeff-bd54-43d1-ace3-b0ad07e82c20 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.257160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd133f-e81f-40eb-866c-ef47d4e6b480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.257824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd133f-e81f-40eb-866c-ef47d4e6b480 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.258152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd133f-e81f-40eb-866c-ef47d4e6b480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.259123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd133f-e81f-40eb-866c-ef47d4e6b480 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.259123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd133f-e81f-40eb-866c-ef47d4e6b480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.261413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd133f-e81f-40eb-866c-ef47d4e6b480 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.269084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 796/1577] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.269984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc3e87-b5e5-40ac-9528-e87aad471d89 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.279980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc3e87-b5e5-40ac-9528-e87aad471d89 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.285378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2621598-dd5c-49cc-b7bc-2376798cfdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.286026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2621598-dd5c-49cc-b7bc-2376798cfdff admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.286310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2621598-dd5c-49cc-b7bc-2376798cfdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.286855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2621598-dd5c-49cc-b7bc-2376798cfdff admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.287236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2621598-dd5c-49cc-b7bc-2376798cfdff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.288851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2621598-dd5c-49cc-b7bc-2376798cfdff admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.292824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 782/1578] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.294235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799b66-0dbb-42f6-91e3-331455404f2d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.294526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799b66-0dbb-42f6-91e3-331455404f2d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.295280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799b66-0dbb-42f6-91e3-331455404f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.302260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 797/1579] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.305863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2621598-dd5c-49cc-b7bc-2376798cfdff admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.317251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2621598-dd5c-49cc-b7bc-2376798cfdff admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.326950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-84799b66-0dbb-42f6-91e3-331455404f2d None None] MFA Rules not processed for user `82edf464a87948c9a806b5986b7e014b`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:27.328125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-84799b66-0dbb-42f6-91e3-331455404f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:27.385922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13f4d3-0858-4764-bd1f-f771a2af39e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.386547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13f4d3-0858-4764-bd1f-f771a2af39e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.386831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13f4d3-0858-4764-bd1f-f771a2af39e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.387213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13f4d3-0858-4764-bd1f-f771a2af39e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.388007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13f4d3-0858-4764-bd1f-f771a2af39e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.389671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13f4d3-0858-4764-bd1f-f771a2af39e7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.401741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:27.411145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 783/1580] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:27.412078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84799b66-0dbb-42f6-91e3-331455404f2d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.417576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 798/1581] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.421514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13f4d3-0858-4764-bd1f-f771a2af39e7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.425082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84799b66-0dbb-42f6-91e3-331455404f2d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.430019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545059e-5bef-4267-a86c-801cde22c2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.430845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545059e-5bef-4267-a86c-801cde22c2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.431245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545059e-5bef-4267-a86c-801cde22c2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.431603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545059e-5bef-4267-a86c-801cde22c2d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.432336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545059e-5bef-4267-a86c-801cde22c2d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.433276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13f4d3-0858-4764-bd1f-f771a2af39e7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.434682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545059e-5bef-4267-a86c-801cde22c2d1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.439379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287d0c-1714-41e4-a44a-f54a1473671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.440191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287d0c-1714-41e4-a44a-f54a1473671f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.440570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287d0c-1714-41e4-a44a-f54a1473671f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.441011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287d0c-1714-41e4-a44a-f54a1473671f admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.441905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f287d0c-1714-41e4-a44a-f54a1473671f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.444219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f287d0c-1714-41e4-a44a-f54a1473671f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.460003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 784/1582] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.461004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545059e-5bef-4267-a86c-801cde22c2d1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.478404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545059e-5bef-4267-a86c-801cde22c2d1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.483628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81680772-7198-4669-a8ff-c8ce45e92001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.484298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81680772-7198-4669-a8ff-c8ce45e92001 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.484706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81680772-7198-4669-a8ff-c8ce45e92001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.485204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81680772-7198-4669-a8ff-c8ce45e92001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.485328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 799/1583] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/users => generated 297 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.486037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81680772-7198-4669-a8ff-c8ce45e92001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.486166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287d0c-1714-41e4-a44a-f54a1473671f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.487686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81680772-7198-4669-a8ff-c8ce45e92001 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.494746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:27.496686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287d0c-1714-41e4-a44a-f54a1473671f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.505286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 785/1584] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.505502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b51e1-7a61-4179-8e87-c39c7bde82ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.505630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81680772-7198-4669-a8ff-c8ce45e92001 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.505730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b51e1-7a61-4179-8e87-c39c7bde82ee admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.505730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b51e1-7a61-4179-8e87-c39c7bde82ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.505730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b51e1-7a61-4179-8e87-c39c7bde82ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.505730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b51e1-7a61-4179-8e87-c39c7bde82ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.506700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b51e1-7a61-4179-8e87-c39c7bde82ee admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.519983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 800/1585] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.520575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81680772-7198-4669-a8ff-c8ce45e92001 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.520863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b51e1-7a61-4179-8e87-c39c7bde82ee admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.529785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686847f9-2020-4166-a79c-aceacff2654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.530507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686847f9-2020-4166-a79c-aceacff2654e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.530812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686847f9-2020-4166-a79c-aceacff2654e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.531133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686847f9-2020-4166-a79c-aceacff2654e admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.531770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686847f9-2020-4166-a79c-aceacff2654e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.534553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686847f9-2020-4166-a79c-aceacff2654e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.536966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b51e1-7a61-4179-8e87-c39c7bde82ee admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.542062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc8c5e-cb67-4ce9-b147-74d5f912215e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.542759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc8c5e-cb67-4ce9-b147-74d5f912215e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.543113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc8c5e-cb67-4ce9-b147-74d5f912215e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.543475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc8c5e-cb67-4ce9-b147-74d5f912215e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.544132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc8c5e-cb67-4ce9-b147-74d5f912215e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.545841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc8c5e-cb67-4ce9-b147-74d5f912215e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.558452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 786/1587] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/users => generated 290 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:27.559179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686847f9-2020-4166-a79c-aceacff2654e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.560828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 801/1586] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.562970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4d8c9-87a0-4c26-ac97-e91f12161b8c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.563412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4d8c9-87a0-4c26-ac97-e91f12161b8c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.563916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4d8c9-87a0-4c26-ac97-e91f12161b8c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.565404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 802/1588] 10.222.0.58 () {54 vars in 961 bytes} [Mon Oct 3 13:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:30:27.566005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4d8c9-87a0-4c26-ac97-e91f12161b8c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.571013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686847f9-2020-4166-a79c-aceacff2654e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.576339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf20b-2157-4e47-9a02-80383a980a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.576948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf20b-2157-4e47-9a02-80383a980a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.577228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf20b-2157-4e47-9a02-80383a980a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.577545 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf20b-2157-4e47-9a02-80383a980a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.578227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abf20b-2157-4e47-9a02-80383a980a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.579621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4d8c9-87a0-4c26-ac97-e91f12161b8c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.579839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abf20b-2157-4e47-9a02-80383a980a91 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.584351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ee327-8c7f-418a-8189-e065e2a7ffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.584962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ee327-8c7f-418a-8189-e065e2a7ffec admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.585250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ee327-8c7f-418a-8189-e065e2a7ffec admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.585574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ee327-8c7f-418a-8189-e065e2a7ffec admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.586230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ee327-8c7f-418a-8189-e065e2a7ffec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.587846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ee327-8c7f-418a-8189-e065e2a7ffec admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.590869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 787/1589] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.591720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf20b-2157-4e47-9a02-80383a980a91 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.601914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 803/1590] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.601914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ee327-8c7f-418a-8189-e065e2a7ffec admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.605474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf20b-2157-4e47-9a02-80383a980a91 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.610264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ee327-8c7f-418a-8189-e065e2a7ffec admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.611392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd1b44-4841-4f1f-adda-6050752570d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.612007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd1b44-4841-4f1f-adda-6050752570d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.612292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd1b44-4841-4f1f-adda-6050752570d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.612570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd1b44-4841-4f1f-adda-6050752570d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.613490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd1b44-4841-4f1f-adda-6050752570d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.615092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd1b44-4841-4f1f-adda-6050752570d4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.615437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec4714-c473-4a9c-9330-92ddfe33d998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.616045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec4714-c473-4a9c-9330-92ddfe33d998 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.616335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec4714-c473-4a9c-9330-92ddfe33d998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.616617 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec4714-c473-4a9c-9330-92ddfe33d998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.617266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec4714-c473-4a9c-9330-92ddfe33d998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.618906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec4714-c473-4a9c-9330-92ddfe33d998 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.626252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 788/1591] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.627099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd1b44-4841-4f1f-adda-6050752570d4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.633292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 804/1592] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.634255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec4714-c473-4a9c-9330-92ddfe33d998 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.642400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd1b44-4841-4f1f-adda-6050752570d4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.649251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec4714-c473-4a9c-9330-92ddfe33d998 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.652668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502576f0-682d-4ed6-8260-846a99ae2932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.653200 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502576f0-682d-4ed6-8260-846a99ae2932 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.653534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502576f0-682d-4ed6-8260-846a99ae2932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.653818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502576f0-682d-4ed6-8260-846a99ae2932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.654480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502576f0-682d-4ed6-8260-846a99ae2932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.655138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0103206-9a42-4d42-bd15-a03e4d4afbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.655912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0103206-9a42-4d42-bd15-a03e4d4afbb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.656711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502576f0-682d-4ed6-8260-846a99ae2932 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.656965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0103206-9a42-4d42-bd15-a03e4d4afbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.657388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0103206-9a42-4d42-bd15-a03e4d4afbb0 admin admin] PATH_INFO: `/v3/projects/d8e6dd58ff43419fa5b945bd08ef2931/users/98de1ccbabc64162a29cbce2cfe11600/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.671036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 789/1593] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.671036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502576f0-682d-4ed6-8260-846a99ae2932 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.674980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0103206-9a42-4d42-bd15-a03e4d4afbb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e6dd58ff43419fa5b945bd08ef2931, user_id=98de1ccbabc64162a29cbce2cfe11600, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.677849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0103206-9a42-4d42-bd15-a03e4d4afbb0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.683448 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502576f0-682d-4ed6-8260-846a99ae2932 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.689726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bb7b3-f0c6-4b1a-ac8f-cad37707e8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.690354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bb7b3-f0c6-4b1a-ac8f-cad37707e8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.690635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bb7b3-f0c6-4b1a-ac8f-cad37707e8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.690946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bb7b3-f0c6-4b1a-ac8f-cad37707e8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.691556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bb7b3-f0c6-4b1a-ac8f-cad37707e8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.693083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 805/1594] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:27 2022] PUT /identity/v3/projects/d8e6dd58ff43419fa5b945bd08ef2931/users/98de1ccbabc64162a29cbce2cfe11600/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:27.693203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bb7b3-f0c6-4b1a-ac8f-cad37707e8e9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.694455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973434f1-98e1-4104-b95f-b64afa83cee9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.694750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973434f1-98e1-4104-b95f-b64afa83cee9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.695029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973434f1-98e1-4104-b95f-b64afa83cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.707776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 790/1595] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.708728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bb7b3-f0c6-4b1a-ac8f-cad37707e8e9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.722877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bb7b3-f0c6-4b1a-ac8f-cad37707e8e9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.746829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-973434f1-98e1-4104-b95f-b64afa83cee9 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:27.748343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-973434f1-98e1-4104-b95f-b64afa83cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:27.786187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc43a6-f1e0-41c1-ad71-cdb4d1f1f882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.786812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc43a6-f1e0-41c1-ad71-cdb4d1f1f882 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.787098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc43a6-f1e0-41c1-ad71-cdb4d1f1f882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.787372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc43a6-f1e0-41c1-ad71-cdb4d1f1f882 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.788057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc43a6-f1e0-41c1-ad71-cdb4d1f1f882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.789734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc43a6-f1e0-41c1-ad71-cdb4d1f1f882 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.800373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 791/1596] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.801211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc43a6-f1e0-41c1-ad71-cdb4d1f1f882 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.815213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc43a6-f1e0-41c1-ad71-cdb4d1f1f882 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.820262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c47a06-0d68-4346-9e6b-6187a54539a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.820887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c47a06-0d68-4346-9e6b-6187a54539a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.821178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c47a06-0d68-4346-9e6b-6187a54539a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.821442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c47a06-0d68-4346-9e6b-6187a54539a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.822169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c47a06-0d68-4346-9e6b-6187a54539a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.823741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c47a06-0d68-4346-9e6b-6187a54539a4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.830060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 806/1597] 10.222.0.58 () {58 vars in 1058 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:27.830937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973434f1-98e1-4104-b95f-b64afa83cee9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.838710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 792/1598] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.838710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c47a06-0d68-4346-9e6b-6187a54539a4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.842375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973434f1-98e1-4104-b95f-b64afa83cee9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.848209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b3c6b-6646-48fc-9f59-d7f678072799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.848826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3c6b-6646-48fc-9f59-d7f678072799 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.849064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c47a06-0d68-4346-9e6b-6187a54539a4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.849545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3c6b-6646-48fc-9f59-d7f678072799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.849545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b3c6b-6646-48fc-9f59-d7f678072799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.850060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b3c6b-6646-48fc-9f59-d7f678072799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.851689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b3c6b-6646-48fc-9f59-d7f678072799 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.855039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e2c4e-c445-49b0-8742-1c17cd46d6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.855723 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2c4e-c445-49b0-8742-1c17cd46d6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.856007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2c4e-c445-49b0-8742-1c17cd46d6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.856283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e2c4e-c445-49b0-8742-1c17cd46d6cc admin admin] PATH_INFO: `/v3/projects/89e4c543318e4bc4b2d594208c99cadf/users/89a7ee61b25941f7a6310e55d1f3e434/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.863406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 807/1599] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:27.864327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b3c6b-6646-48fc-9f59-d7f678072799 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.873228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e2c4e-c445-49b0-8742-1c17cd46d6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e4c543318e4bc4b2d594208c99cadf, user_id=89a7ee61b25941f7a6310e55d1f3e434, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.875451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b3c6b-6646-48fc-9f59-d7f678072799 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:27.876594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e2c4e-c445-49b0-8742-1c17cd46d6cc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.880976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6c1dc-770b-4acb-a511-f76af01a4923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:27.881706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6c1dc-770b-4acb-a511-f76af01a4923 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.881958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6c1dc-770b-4acb-a511-f76af01a4923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.882251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6c1dc-770b-4acb-a511-f76af01a4923 admin admin] PATH_INFO: `/v3/projects/5243c0442fc04fe7b74708c0facdc3ba/users/96232e94467b4130b98baed628bae061/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.891688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 793/1600] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:27 2022] PUT /identity/v3/projects/89e4c543318e4bc4b2d594208c99cadf/users/89a7ee61b25941f7a6310e55d1f3e434/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:27.892861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73917b39-779f-4d64-945d-a96a8b080b5d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:27.893274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73917b39-779f-4d64-945d-a96a8b080b5d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:27.893699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73917b39-779f-4d64-945d-a96a8b080b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:27.908743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6c1dc-770b-4acb-a511-f76af01a4923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5243c0442fc04fe7b74708c0facdc3ba, user_id=96232e94467b4130b98baed628bae061, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:27.913945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6c1dc-770b-4acb-a511-f76af01a4923 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:27.919652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-73917b39-779f-4d64-945d-a96a8b080b5d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:27.922580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-73917b39-779f-4d64-945d-a96a8b080b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:27.931045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 808/1601] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:27 2022] PUT /identity/v3/projects/5243c0442fc04fe7b74708c0facdc3ba/users/96232e94467b4130b98baed628bae061/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:27.941647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6c1dc-770b-4acb-a511-f76af01a4923 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:27.965264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:28.001356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:28.011333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 794/1602] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:28.012182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73917b39-779f-4d64-945d-a96a8b080b5d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:28.025923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73917b39-779f-4d64-945d-a96a8b080b5d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:28.029147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6c1dc-770b-4acb-a511-f76af01a4923 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:28.030747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f39332-12fa-4cf0-b6fd-2510a1b9b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:28.031532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f39332-12fa-4cf0-b6fd-2510a1b9b8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.032031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f39332-12fa-4cf0-b6fd-2510a1b9b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.032368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f39332-12fa-4cf0-b6fd-2510a1b9b8cd admin admin] PATH_INFO: `/v3/projects/f402f6d4d8964fa1a5c0275873535fa1/users/e7ecab5bc21049e9991d63a79779b7d2/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.034408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b0b4-e078-432f-92fa-077d03b0f06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:28.035101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b0b4-e078-432f-92fa-077d03b0f06e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.035417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b0b4-e078-432f-92fa-077d03b0f06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.035691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b0b4-e078-432f-92fa-077d03b0f06e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.036368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b0b4-e078-432f-92fa-077d03b0f06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:28.038010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b0b4-e078-432f-92fa-077d03b0f06e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:28.047573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f39332-12fa-4cf0-b6fd-2510a1b9b8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f402f6d4d8964fa1a5c0275873535fa1, user_id=e7ecab5bc21049e9991d63a79779b7d2, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:28.048243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 809/1603] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:28.049163 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b0b4-e078-432f-92fa-077d03b0f06e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:28.050315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f39332-12fa-4cf0-b6fd-2510a1b9b8cd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:28.066165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 795/1604] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:28 2022] PUT /identity/v3/projects/f402f6d4d8964fa1a5c0275873535fa1/users/e7ecab5bc21049e9991d63a79779b7d2/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:28.066491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f39332-12fa-4cf0-b6fd-2510a1b9b8cd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:28.136592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f39332-12fa-4cf0-b6fd-2510a1b9b8cd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:28.140954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b0b4-e078-432f-92fa-077d03b0f06e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:28.141694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac06388-83fd-4b82-bcec-8be9883bd700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:28.142366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac06388-83fd-4b82-bcec-8be9883bd700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.142642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac06388-83fd-4b82-bcec-8be9883bd700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.142968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac06388-83fd-4b82-bcec-8be9883bd700 admin admin] PATH_INFO: `/v3/projects/277f039d5372403cae3f766dfd01c81c/users/ddbe4e9452c241bfbeddbdb5e51a865b/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.158744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac06388-83fd-4b82-bcec-8be9883bd700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277f039d5372403cae3f766dfd01c81c, user_id=ddbe4e9452c241bfbeddbdb5e51a865b, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:28.161417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac06388-83fd-4b82-bcec-8be9883bd700 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:28.177992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 796/1605] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:28 2022] PUT /identity/v3/projects/277f039d5372403cae3f766dfd01c81c/users/ddbe4e9452c241bfbeddbdb5e51a865b/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:28.178967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac06388-83fd-4b82-bcec-8be9883bd700 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:28.212803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10526b0-a89a-4d2a-8714-cb5fb5c2da84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:28.213593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10526b0-a89a-4d2a-8714-cb5fb5c2da84 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.214096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10526b0-a89a-4d2a-8714-cb5fb5c2da84 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.214224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10526b0-a89a-4d2a-8714-cb5fb5c2da84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.248826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac06388-83fd-4b82-bcec-8be9883bd700 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:28.254866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adafffe1-b5c6-4fca-aecd-b2068197c2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:28.255543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafffe1-b5c6-4fca-aecd-b2068197c2ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.256248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafffe1-b5c6-4fca-aecd-b2068197c2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.256524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adafffe1-b5c6-4fca-aecd-b2068197c2ed admin admin] PATH_INFO: `/v3/projects/557321bf33ca42df8d5d53e5157e6803/users/ae71fd7ba4334d0d813cbc0c06e3686e/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.259405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:28.272522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafffe1-b5c6-4fca-aecd-b2068197c2ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=557321bf33ca42df8d5d53e5157e6803, user_id=ae71fd7ba4334d0d813cbc0c06e3686e, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:28.275126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10526b0-a89a-4d2a-8714-cb5fb5c2da84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:28.275292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adafffe1-b5c6-4fca-aecd-b2068197c2ed admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:28.276723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10526b0-a89a-4d2a-8714-cb5fb5c2da84 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:28.287557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 810/1606] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:28.289269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77dbce-90db-4fff-b438-db2676f8f1bd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.289590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77dbce-90db-4fff-b438-db2676f8f1bd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.289870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77dbce-90db-4fff-b438-db2676f8f1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.299333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 797/1607] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:28 2022] PUT /identity/v3/projects/557321bf33ca42df8d5d53e5157e6803/users/ae71fd7ba4334d0d813cbc0c06e3686e/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:28.300967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aef14-40e6-4cad-8c0a-f31ba484d14d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.301666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aef14-40e6-4cad-8c0a-f31ba484d14d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.302130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844aef14-40e6-4cad-8c0a-f31ba484d14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.323598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8e77dbce-90db-4fff-b438-db2676f8f1bd None None] MFA Rules not processed for user `89a7ee61b25941f7a6310e55d1f3e434`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:28.324977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8e77dbce-90db-4fff-b438-db2676f8f1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:28.336837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-844aef14-40e6-4cad-8c0a-f31ba484d14d None None] MFA Rules not processed for user `96232e94467b4130b98baed628bae061`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:28.338112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-844aef14-40e6-4cad-8c0a-f31ba484d14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:28.372376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:28.418122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 811/1608] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:28 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:28.419287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77dbce-90db-4fff-b438-db2676f8f1bd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:28.428756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 798/1609] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:28.429919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aef14-40e6-4cad-8c0a-f31ba484d14d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:28.496433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77dbce-90db-4fff-b438-db2676f8f1bd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:28.500969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13759028-88f3-478f-a2ba-0bb3a3034c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:28.501610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13759028-88f3-478f-a2ba-0bb3a3034c4e service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.501893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13759028-88f3-478f-a2ba-0bb3a3034c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.502167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13759028-88f3-478f-a2ba-0bb3a3034c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.552957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844aef14-40e6-4cad-8c0a-f31ba484d14d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:28.566552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cf417-c7aa-421f-8a76-4595a5b3ad83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:28.567250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cf417-c7aa-421f-8a76-4595a5b3ad83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.567536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cf417-c7aa-421f-8a76-4595a5b3ad83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.567815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cf417-c7aa-421f-8a76-4595a5b3ad83 admin admin] PATH_INFO: `/v3/projects/5f908aee304e4fe2bb2185d89400533c/users/b8defdf430274affaed766d8aedccfad/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.586794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13759028-88f3-478f-a2ba-0bb3a3034c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:28.588244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13759028-88f3-478f-a2ba-0bb3a3034c4e service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:28.596417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84cf417-c7aa-421f-8a76-4595a5b3ad83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f908aee304e4fe2bb2185d89400533c, user_id=b8defdf430274affaed766d8aedccfad, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:28.599536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 812/1610] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:28.600761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9e252-b2b6-4104-b7ce-d5fd67e1666f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.601020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9e252-b2b6-4104-b7ce-d5fd67e1666f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.601295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9e252-b2b6-4104-b7ce-d5fd67e1666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.605717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84cf417-c7aa-421f-8a76-4595a5b3ad83 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:28.630341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 799/1611] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:28 2022] PUT /identity/v3/projects/5f908aee304e4fe2bb2185d89400533c/users/b8defdf430274affaed766d8aedccfad/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:28.631763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6213-7210-4981-a094-7c6e1b4a0c20 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.632122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6213-7210-4981-a094-7c6e1b4a0c20 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.632468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad6213-7210-4981-a094-7c6e1b4a0c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.638627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-38b9e252-b2b6-4104-b7ce-d5fd67e1666f None None] MFA Rules not processed for user `e7ecab5bc21049e9991d63a79779b7d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:28.640160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-38b9e252-b2b6-4104-b7ce-d5fd67e1666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:28.652126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:28.664127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-88ad6213-7210-4981-a094-7c6e1b4a0c20 None None] MFA Rules not processed for user `ddbe4e9452c241bfbeddbdb5e51a865b`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:28.665541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-88ad6213-7210-4981-a094-7c6e1b4a0c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:28.723754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 813/1612] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:28.725863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd6dea-8f53-4881-87f5-7bd31b5771d4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.725863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd6dea-8f53-4881-87f5-7bd31b5771d4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.725863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd6dea-8f53-4881-87f5-7bd31b5771d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.749295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-68bd6dea-8f53-4881-87f5-7bd31b5771d4 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:28.751483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-68bd6dea-8f53-4881-87f5-7bd31b5771d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:28.754582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 800/1613] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:28.756124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad6213-7210-4981-a094-7c6e1b4a0c20 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:28.825155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:28.851185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad6213-7210-4981-a094-7c6e1b4a0c20 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:28.862601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352495e-76d5-4e07-b4cc-5c768619bf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:28.863247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352495e-76d5-4e07-b4cc-5c768619bf8e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.863586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352495e-76d5-4e07-b4cc-5c768619bf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.864009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352495e-76d5-4e07-b4cc-5c768619bf8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.864670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352495e-76d5-4e07-b4cc-5c768619bf8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:28.870426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 814/1614] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:28.871189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352495e-76d5-4e07-b4cc-5c768619bf8e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:28.872092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea38f0-158a-4baa-a038-03e9b137399f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.872092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea38f0-158a-4baa-a038-03e9b137399f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.872092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea38f0-158a-4baa-a038-03e9b137399f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.872703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 815/1615] 10.222.0.58 () {54 vars in 961 bytes} [Mon Oct 3 13:30:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:30:28.875991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867e375-04da-4d7c-988a-68f6337823b1 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.877250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867e375-04da-4d7c-988a-68f6337823b1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.877718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867e375-04da-4d7c-988a-68f6337823b1 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.880825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 816/1616] 10.222.0.58 () {54 vars in 912 bytes} [Mon Oct 3 13:30:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:30:28.883019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5daa4-ad4d-46c7-bd3d-7fc4b0e59ddd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:28.884467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5daa4-ad4d-46c7-bd3d-7fc4b0e59ddd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:28.885051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5daa4-ad4d-46c7-bd3d-7fc4b0e59ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:28.916585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1bf5daa4-ad4d-46c7-bd3d-7fc4b0e59ddd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:28.918546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1bf5daa4-ad4d-46c7-bd3d-7fc4b0e59ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:28.923161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 801/1617] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:28 2022] POST /identity/v3/projects => generated 403 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:28.924586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352495e-76d5-4e07-b4cc-5c768619bf8e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.023815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352495e-76d5-4e07-b4cc-5c768619bf8e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.028208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 817/1618] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:29.028208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5daa4-ad4d-46c7-bd3d-7fc4b0e59ddd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.029795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1bf5daa4-ad4d-46c7-bd3d-7fc4b0e59ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:29.034254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dc416-4d69-4b85-a521-939b07aa001c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.035469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc416-4d69-4b85-a521-939b07aa001c service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.035962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc416-4d69-4b85-a521-939b07aa001c service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.036410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc416-4d69-4b85-a521-939b07aa001c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.146643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5daa4-ad4d-46c7-bd3d-7fc4b0e59ddd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.153443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8ff6-3839-4afa-b49b-6ef75142251b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.155257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8ff6-3839-4afa-b49b-6ef75142251b service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.157607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8ff6-3839-4afa-b49b-6ef75142251b service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.158047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc416-4d69-4b85-a521-939b07aa001c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.158167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8ff6-3839-4afa-b49b-6ef75142251b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.161230 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc416-4d69-4b85-a521-939b07aa001c service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.171196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:29.178728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b8ff6-3839-4afa-b49b-6ef75142251b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.181499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b8ff6-3839-4afa-b49b-6ef75142251b service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.185961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 802/1619] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:29.189848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527213e-9828-4513-97be-f0b4c8c6b8c7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.189848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527213e-9828-4513-97be-f0b4c8c6b8c7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.190031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527213e-9828-4513-97be-f0b4c8c6b8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.210307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 818/1620] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:29.212870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a1a62-91c1-4953-abc8-68d31a60ea95 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.213367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a1a62-91c1-4953-abc8-68d31a60ea95 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.214276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a1a62-91c1-4953-abc8-68d31a60ea95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.230876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6527213e-9828-4513-97be-f0b4c8c6b8c7 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:29.234471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6527213e-9828-4513-97be-f0b4c8c6b8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:29.254871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c67a1a62-91c1-4953-abc8-68d31a60ea95 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:29.256442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c67a1a62-91c1-4953-abc8-68d31a60ea95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:29.339922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 803/1621] 10.222.0.58 () {58 vars in 1058 bytes} [Mon Oct 3 13:30:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:29.341244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527213e-9828-4513-97be-f0b4c8c6b8c7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.359803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527213e-9828-4513-97be-f0b4c8c6b8c7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.365309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf676c6a-4124-4553-89ea-2f0fbd1d7b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.366080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf676c6a-4124-4553-89ea-2f0fbd1d7b28 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.366472 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf676c6a-4124-4553-89ea-2f0fbd1d7b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.366830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf676c6a-4124-4553-89ea-2f0fbd1d7b28 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.367571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf676c6a-4124-4553-89ea-2f0fbd1d7b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.369827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf676c6a-4124-4553-89ea-2f0fbd1d7b28 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.380817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 819/1622] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:30:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:29.381726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a1a62-91c1-4953-abc8-68d31a60ea95 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.396383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a1a62-91c1-4953-abc8-68d31a60ea95 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.401818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:29.403183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ae9cb-612d-45d4-8129-ccfd75152766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.404605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae9cb-612d-45d4-8129-ccfd75152766 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.404927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae9cb-612d-45d4-8129-ccfd75152766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.405667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ae9cb-612d-45d4-8129-ccfd75152766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.417249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 804/1623] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:29 2022] POST /identity/v3/users => generated 308 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:29.417249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf676c6a-4124-4553-89ea-2f0fbd1d7b28 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.430192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ae9cb-612d-45d4-8129-ccfd75152766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.432115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ae9cb-612d-45d4-8129-ccfd75152766 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.434469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf676c6a-4124-4553-89ea-2f0fbd1d7b28 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.442496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4879b7-cf18-4a54-ac63-4924feccdb1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.443183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4879b7-cf18-4a54-ac63-4924feccdb1f service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.443467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4879b7-cf18-4a54-ac63-4924feccdb1f service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.443746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4879b7-cf18-4a54-ac63-4924feccdb1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.447294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 820/1624] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:29.448261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ae9cb-612d-45d4-8129-ccfd75152766 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.458998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ae9cb-612d-45d4-8129-ccfd75152766 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.467285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f653-193f-426d-8f1d-eb96a9f13481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.468199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f653-193f-426d-8f1d-eb96a9f13481 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.469517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f653-193f-426d-8f1d-eb96a9f13481 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.470966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496f653-193f-426d-8f1d-eb96a9f13481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.495449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496f653-193f-426d-8f1d-eb96a9f13481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.497102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496f653-193f-426d-8f1d-eb96a9f13481 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.517702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 821/1625] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:29.518853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f653-193f-426d-8f1d-eb96a9f13481 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.543821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4879b7-cf18-4a54-ac63-4924feccdb1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.545933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4879b7-cf18-4a54-ac63-4924feccdb1f service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.546652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496f653-193f-426d-8f1d-eb96a9f13481 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.560439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 805/1626] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:29.577004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec35fdc-f914-4694-a7bc-8de73a01d29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.577668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35fdc-f914-4694-a7bc-8de73a01d29b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.578130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35fdc-f914-4694-a7bc-8de73a01d29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.578834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec35fdc-f914-4694-a7bc-8de73a01d29b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.579216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec35fdc-f914-4694-a7bc-8de73a01d29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.581037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec35fdc-f914-4694-a7bc-8de73a01d29b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.613132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 822/1627] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:29.621971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4879b7-cf18-4a54-ac63-4924feccdb1f service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.622296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec35fdc-f914-4694-a7bc-8de73a01d29b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.643275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4879b7-cf18-4a54-ac63-4924feccdb1f service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.654174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec35fdc-f914-4694-a7bc-8de73a01d29b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.661090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2fa9a-e9b8-4046-9d09-2fcc39f2f82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.663254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2fa9a-e9b8-4046-9d09-2fcc39f2f82f admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.664803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2fa9a-e9b8-4046-9d09-2fcc39f2f82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.664803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2fa9a-e9b8-4046-9d09-2fcc39f2f82f admin admin] PATH_INFO: `/v3/projects/5e3b1f825440440c93c13e183cc58f0e/users/c8db184b6f604786a8d7342d2e2f2de9/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.672929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799adfa-1694-40f0-b4b3-ea9d21196934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.674504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799adfa-1694-40f0-b4b3-ea9d21196934 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.675768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799adfa-1694-40f0-b4b3-ea9d21196934 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.676120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799adfa-1694-40f0-b4b3-ea9d21196934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.703763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2fa9a-e9b8-4046-9d09-2fcc39f2f82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3b1f825440440c93c13e183cc58f0e, user_id=c8db184b6f604786a8d7342d2e2f2de9, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.710347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2fa9a-e9b8-4046-9d09-2fcc39f2f82f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.742282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 806/1628] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:29 2022] PUT /identity/v3/projects/5e3b1f825440440c93c13e183cc58f0e/users/c8db184b6f604786a8d7342d2e2f2de9/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:29.746391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2fa9a-e9b8-4046-9d09-2fcc39f2f82f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.837970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799adfa-1694-40f0-b4b3-ea9d21196934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.843341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799adfa-1694-40f0-b4b3-ea9d21196934 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.896407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 823/1629] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:29.897069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2fa9a-e9b8-4046-9d09-2fcc39f2f82f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.904811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaaba6-5713-4e86-a48f-55bdbea277d8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.905137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaaba6-5713-4e86-a48f-55bdbea277d8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.906795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaaba6-5713-4e86-a48f-55bdbea277d8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.914038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 824/1630] 10.222.0.58 () {54 vars in 912 bytes} [Mon Oct 3 13:30:29 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:30:29.922348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada41e6-8bee-460f-a57c-0df421ecd69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.923287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bce0e-c71b-44b9-a9b9-b17d50a40bc1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.924215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada41e6-8bee-460f-a57c-0df421ecd69d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.924725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada41e6-8bee-460f-a57c-0df421ecd69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.925201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada41e6-8bee-460f-a57c-0df421ecd69d admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.925419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bce0e-c71b-44b9-a9b9-b17d50a40bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.925743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bce0e-c71b-44b9-a9b9-b17d50a40bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.927532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada41e6-8bee-460f-a57c-0df421ecd69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:29.930145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada41e6-8bee-460f-a57c-0df421ecd69d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:29.952157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 807/1631] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:29.953798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada41e6-8bee-460f-a57c-0df421ecd69d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:29.968473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada41e6-8bee-460f-a57c-0df421ecd69d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:29.983056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209755eb-0dff-4751-a1ff-1e23f0b9d386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:29.983765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209755eb-0dff-4751-a1ff-1e23f0b9d386 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:29.984095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209755eb-0dff-4751-a1ff-1e23f0b9d386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:29.984382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209755eb-0dff-4751-a1ff-1e23f0b9d386 admin admin] PATH_INFO: `/v3/projects/5e3b1f825440440c93c13e183cc58f0e/users/c8db184b6f604786a8d7342d2e2f2de9/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:29.986010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-df0bce0e-c71b-44b9-a9b9-b17d50a40bc1 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:29.989659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-df0bce0e-c71b-44b9-a9b9-b17d50a40bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:30.003077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209755eb-0dff-4751-a1ff-1e23f0b9d386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3b1f825440440c93c13e183cc58f0e, user_id=c8db184b6f604786a8d7342d2e2f2de9, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:30.008274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209755eb-0dff-4751-a1ff-1e23f0b9d386 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:30.049467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 808/1632] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:29 2022] PUT /identity/v3/projects/5e3b1f825440440c93c13e183cc58f0e/users/c8db184b6f604786a8d7342d2e2f2de9/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:30.062466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8be78-c1ca-4758-adf8-ff38301f955c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:30.063114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8be78-c1ca-4758-adf8-ff38301f955c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:30.063556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8be78-c1ca-4758-adf8-ff38301f955c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:30.122849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:30.147172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ddd8be78-c1ca-4758-adf8-ff38301f955c None None] MFA Rules not processed for user `c8db184b6f604786a8d7342d2e2f2de9`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:30.148440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ddd8be78-c1ca-4758-adf8-ff38301f955c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:30.203562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 825/1633] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:30:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:30.217684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bce0e-c71b-44b9-a9b9-b17d50a40bc1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:30.304179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:30.365848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bce0e-c71b-44b9-a9b9-b17d50a40bc1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:30.375912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 809/1634] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:30:30.482203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc7c63-1020-4083-b14d-f6185ee8bbaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:30.483181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc7c63-1020-4083-b14d-f6185ee8bbaa service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:30.483640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc7c63-1020-4083-b14d-f6185ee8bbaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:30.485332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc7c63-1020-4083-b14d-f6185ee8bbaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:30.622981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc7c63-1020-4083-b14d-f6185ee8bbaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:30.625728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc7c63-1020-4083-b14d-f6185ee8bbaa service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:30.642228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 826/1635] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:30.954860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8be78-c1ca-4758-adf8-ff38301f955c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:30.967473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8be78-c1ca-4758-adf8-ff38301f955c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:30.976326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be9275-14cb-46e2-85d2-bee495571bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:30.977947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be9275-14cb-46e2-85d2-bee495571bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:30.978241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be9275-14cb-46e2-85d2-bee495571bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:30.978526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be9275-14cb-46e2-85d2-bee495571bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:31.005817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be9275-14cb-46e2-85d2-bee495571bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:31.007301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be9275-14cb-46e2-85d2-bee495571bc0 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:31.044181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 810/1636] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:30 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:31.814292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc7c63-1020-4083-b14d-f6185ee8bbaa service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:31.966777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:32.043502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be9275-14cb-46e2-85d2-bee495571bc0 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:32.055111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc7c63-1020-4083-b14d-f6185ee8bbaa service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:32.059538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be9275-14cb-46e2-85d2-bee495571bc0 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:32.060735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:32.061404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:32.061728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:32.062852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] PATH_INFO: `/v3/users/89a7ee61b25941f7a6310e55d1f3e434` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:32.064485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a7ee61b25941f7a6310e55d1f3e434)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:32.066621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:32.083342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:32.084075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:32.084303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:32.084571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] PATH_INFO: `/v3/users/82edf464a87948c9a806b5986b7e014b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:32.087009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82edf464a87948c9a806b5986b7e014b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:32.088067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: WARNING py.warnings [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:30:32.088067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: warnings.warn(msg) Oct 03 13:30:32.088067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: Oct 03 13:30:32.089306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:32.163731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a7ee61b25941f7a6310e55d1f3e434'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.184667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a7ee61b25941f7a6310e55d1f3e434'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.185934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a7ee61b25941f7a6310e55d1f3e434'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.200473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a7ee61b25941f7a6310e55d1f3e434'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.220908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-115fd903-cac7-4f67-a1d5-674282337fcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a7ee61b25941f7a6310e55d1f3e434'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.231708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82edf464a87948c9a806b5986b7e014b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.232401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82edf464a87948c9a806b5986b7e014b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.233837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82edf464a87948c9a806b5986b7e014b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.238556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 827/1637] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:31 2022] DELETE /identity/v3/users/89a7ee61b25941f7a6310e55d1f3e434 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:32.243940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac612e-bc48-4339-a400-3a5e5808b068 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:32.244333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac612e-bc48-4339-a400-3a5e5808b068 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:32.244788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac612e-bc48-4339-a400-3a5e5808b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:32.273210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82edf464a87948c9a806b5986b7e014b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.277805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-98ac612e-bc48-4339-a400-3a5e5808b068 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:32.279322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-98ac612e-bc48-4339-a400-3a5e5808b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:32.370599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82edf464a87948c9a806b5986b7e014b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:32.404111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 828/1638] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:32.436060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 811/1639] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:32 2022] DELETE /identity/v3/users/82edf464a87948c9a806b5986b7e014b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:32.441177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33a5b4-f373-422b-89f6-a0794eceb08a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:32.441574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33a5b4-f373-422b-89f6-a0794eceb08a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:32.441788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33a5b4-f373-422b-89f6-a0794eceb08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:32.444377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:32.446639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:32.488771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fa33a5b4-f373-422b-89f6-a0794eceb08a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:32.490281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fa33a5b4-f373-422b-89f6-a0794eceb08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:32.651148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:32.710624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 829/1640] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:30:32.722266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a7ce9-cf50-48a6-98fb-b5366f914192 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:32.761289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33a5b4-f373-422b-89f6-a0794eceb08a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:32.763129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fa33a5b4-f373-422b-89f6-a0794eceb08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:32.766617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:32.777233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:32.781925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:32.788726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:32.796106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33a5b4-f373-422b-89f6-a0794eceb08a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:32.846448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a12e3-fe47-47e6-8668-3e6f51adbb08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:32.847207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a12e3-fe47-47e6-8668-3e6f51adbb08 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:32.847410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a12e3-fe47-47e6-8668-3e6f51adbb08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:32.848448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a12e3-fe47-47e6-8668-3e6f51adbb08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:32.873855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:32.881887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:32.890341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a12e3-fe47-47e6-8668-3e6f51adbb08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:32.891838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a12e3-fe47-47e6-8668-3e6f51adbb08 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:32.941161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 812/1641] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:30:32.950406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 830/1642] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:30:33.917007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:33.919575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:33.948434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64401e-086c-411f-9b2e-ed65e1704648 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:33.970641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:33.971494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:33.971786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:33.972067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] PATH_INFO: `/v3/projects/89e4c543318e4bc4b2d594208c99cadf` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:33.974818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e4c543318e4bc4b2d594208c99cadf)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:33.976629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:34.189097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:34.213703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e4c543318e4bc4b2d594208c99cadf', 'request_id': 'req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.231421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e4c543318e4bc4b2d594208c99cadf', 'request_id': 'req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.245580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e4c543318e4bc4b2d594208c99cadf', 'request_id': 'req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.282726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] The token cache is being invalidate because project 89e4c543318e4bc4b2d594208c99cadf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:30:34.283678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.285743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.301527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 813/1643] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:33 2022] DELETE /identity/v3/projects/89e4c543318e4bc4b2d594208c99cadf => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:34.310402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a12e3-fe47-47e6-8668-3e6f51adbb08 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:34.364446 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a12e3-fe47-47e6-8668-3e6f51adbb08 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:34.608073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:34.613060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:34.636291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:34.638678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:34.638964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:34.639284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] PATH_INFO: `/v3/users/c8db184b6f604786a8d7342d2e2f2de9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:34.643678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8db184b6f604786a8d7342d2e2f2de9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:34.645887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:34.677710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8c6fc-73f8-4a36-b960-50ff1b5ac331 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:34.720752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:34.721429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:34.723689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:34.723989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] PATH_INFO: `/v3/users/96232e94467b4130b98baed628bae061` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:34.737326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96232e94467b4130b98baed628bae061)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:34.748198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:34.845401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8db184b6f604786a8d7342d2e2f2de9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.879917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8db184b6f604786a8d7342d2e2f2de9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.880873 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8db184b6f604786a8d7342d2e2f2de9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.897934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8db184b6f604786a8d7342d2e2f2de9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.930571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b6f3e95f-0ab9-443e-bc86-1094105d0d81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8db184b6f604786a8d7342d2e2f2de9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.950050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 831/1644] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:34 2022] DELETE /identity/v3/users/c8db184b6f604786a8d7342d2e2f2de9 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:34.975382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96232e94467b4130b98baed628bae061'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.976177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96232e94467b4130b98baed628bae061'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:34.976841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96232e94467b4130b98baed628bae061'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:35.006977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96232e94467b4130b98baed628bae061'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:35.070260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96232e94467b4130b98baed628bae061'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:35.107396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 814/1645] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:34 2022] DELETE /identity/v3/users/96232e94467b4130b98baed628bae061 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:35.114165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b798f-574c-4b25-becf-ec81b72a8e97 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:35.114834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b798f-574c-4b25-becf-ec81b72a8e97 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:35.115347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b798f-574c-4b25-becf-ec81b72a8e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:35.175178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-de2b798f-574c-4b25-becf-ec81b72a8e97 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:35.179719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-de2b798f-574c-4b25-becf-ec81b72a8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:35.460677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 832/1646] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:35.815090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:35.817297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:35.944538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62524c-6894-492a-b978-f38dae5623d8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:35.962237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085284-a11b-4cdc-b889-304989e9e334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:35.964029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085284-a11b-4cdc-b889-304989e9e334 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:35.968737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085284-a11b-4cdc-b889-304989e9e334 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:35.969582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085284-a11b-4cdc-b889-304989e9e334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:35.993545 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085284-a11b-4cdc-b889-304989e9e334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:35.995809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085284-a11b-4cdc-b889-304989e9e334 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:36.027290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 815/1647] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:36.252426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b798f-574c-4b25-becf-ec81b72a8e97 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:36.255174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-de2b798f-574c-4b25-becf-ec81b72a8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:36.275782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:36.289480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b798f-574c-4b25-becf-ec81b72a8e97 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:36.320802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:36.322366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:36.322754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:36.323052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] PATH_INFO: `/v3/projects/3d84507eb30f4a7fabd41f8962a98e06` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:36.329733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d84507eb30f4a7fabd41f8962a98e06)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:36.330946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 03 13:30:36.330946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 03 13:30:36.330946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: Oct 03 13:30:36.332352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:36.444815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d84507eb30f4a7fabd41f8962a98e06', 'request_id': 'req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:36.457170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d84507eb30f4a7fabd41f8962a98e06', 'request_id': 'req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:36.480062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d84507eb30f4a7fabd41f8962a98e06', 'request_id': 'req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:36.513741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] The token cache is being invalidate because project 3d84507eb30f4a7fabd41f8962a98e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:30:36.513964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:36.516237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b5296ef9-5d2e-4cca-a22e-69bfbd547f1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:36.518378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 833/1648] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:36 2022] DELETE /identity/v3/projects/3d84507eb30f4a7fabd41f8962a98e06 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:36.526524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937c237-c143-488e-a136-cba2695c6680 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:36.527414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937c237-c143-488e-a136-cba2695c6680 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:36.528228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937c237-c143-488e-a136-cba2695c6680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:36.601420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0937c237-c143-488e-a136-cba2695c6680 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:36.603615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0937c237-c143-488e-a136-cba2695c6680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:36.697986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:36.804423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 816/1649] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:36.810561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60805c7e-eb46-4c94-8939-d4994d5c2043 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:36.810561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60805c7e-eb46-4c94-8939-d4994d5c2043 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:36.810561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60805c7e-eb46-4c94-8939-d4994d5c2043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:36.873500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-60805c7e-eb46-4c94-8939-d4994d5c2043 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:36.885349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-60805c7e-eb46-4c94-8939-d4994d5c2043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:36.936552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937c237-c143-488e-a136-cba2695c6680 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:36.951772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 834/1650] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:30:36.976305 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937c237-c143-488e-a136-cba2695c6680 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:37.008357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:37.009777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:37.011436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:37.012307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:37.015514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:37.016081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:37.019698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:37.101122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60805c7e-eb46-4c94-8939-d4994d5c2043 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:37.120610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 817/1651] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:37.123147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60805c7e-eb46-4c94-8939-d4994d5c2043 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:37.185385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:37.186449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:37.192346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:37.192840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:37.194247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:37.207020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:37.367238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 835/1652] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:37 2022] POST /identity/v3/projects => generated 391 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:37.371359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:37.383773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:37.386453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:37.413207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253483ca-247a-4840-980a-2e82f8a75a8a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:37.696454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0bc5c-7d11-498a-bd66-a7f59f5e9bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:37.697113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0bc5c-7d11-498a-bd66-a7f59f5e9bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:37.697401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0bc5c-7d11-498a-bd66-a7f59f5e9bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:37.700551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0bc5c-7d11-498a-bd66-a7f59f5e9bbe admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:37.701338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0bc5c-7d11-498a-bd66-a7f59f5e9bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:37.713045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b0bc5c-7d11-498a-bd66-a7f59f5e9bbe admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:37.715578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7ed49-d7ee-4fc2-a6bb-403fdac20ba2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:37.762026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:37.767284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:37.770243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:37.779058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] PATH_INFO: `/v3/projects/5e3b1f825440440c93c13e183cc58f0e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:37.785202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3b1f825440440c93c13e183cc58f0e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:37.788020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:37.795315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 818/1653] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:37 2022] POST /identity/v3/users => generated 296 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:30:37.805633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0bc5c-7d11-498a-bd66-a7f59f5e9bbe admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:37.831983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0bc5c-7d11-498a-bd66-a7f59f5e9bbe admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:37.865317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462e6d-2eeb-4db4-b450-5a1abd24af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:37.866013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462e6d-2eeb-4db4-b450-5a1abd24af9e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:37.866305 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462e6d-2eeb-4db4-b450-5a1abd24af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:37.866607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462e6d-2eeb-4db4-b450-5a1abd24af9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:37.867276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462e6d-2eeb-4db4-b450-5a1abd24af9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:37.869015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462e6d-2eeb-4db4-b450-5a1abd24af9e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:37.917170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 819/1654] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:37.923968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462e6d-2eeb-4db4-b450-5a1abd24af9e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:37.973749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462e6d-2eeb-4db4-b450-5a1abd24af9e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:37.984525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3b1f825440440c93c13e183cc58f0e', 'request_id': 'req-8c9522b3-282d-4944-920d-27b10d81332f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:38.033378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3b1f825440440c93c13e183cc58f0e', 'request_id': 'req-8c9522b3-282d-4944-920d-27b10d81332f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:38.040554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:38.058812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3b1f825440440c93c13e183cc58f0e', 'request_id': 'req-8c9522b3-282d-4944-920d-27b10d81332f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:38.090752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] The token cache is being invalidate because project 5e3b1f825440440c93c13e183cc58f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:30:38.091153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:38.095904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8c9522b3-282d-4944-920d-27b10d81332f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:38.105697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 836/1655] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:37 2022] DELETE /identity/v3/projects/5e3b1f825440440c93c13e183cc58f0e => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:30:38.109702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ed27-3f23-40ef-be19-1b7408624e44 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:38.109702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ed27-3f23-40ef-be19-1b7408624e44 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:38.109702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ed27-3f23-40ef-be19-1b7408624e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:38.176151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-56e2ed27-3f23-40ef-be19-1b7408624e44 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:38.183855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-56e2ed27-3f23-40ef-be19-1b7408624e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:38.226451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:38.227627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:38.227933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:38.228209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:38.228903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:38.235883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:38.279810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 820/1656] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:30:38.287401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:38.298488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:38.330652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007044b-286c-4994-91b3-670e3ca338a0 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:38.366865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee82dae-f8d1-4a07-9319-24eb5407805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:38.367635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82dae-f8d1-4a07-9319-24eb5407805e admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:38.367964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82dae-f8d1-4a07-9319-24eb5407805e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:38.368247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82dae-f8d1-4a07-9319-24eb5407805e admin admin] PATH_INFO: `/v3/projects/2c610147005e48a3801c5f5f23067f11/users/26fa926d96ee40e69af516af455ecda0/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:38.453849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 837/1657] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:38.456970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94be614-47d9-47f5-a21d-b88d1e12e3aa None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:38.457266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94be614-47d9-47f5-a21d-b88d1e12e3aa None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:38.457884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94be614-47d9-47f5-a21d-b88d1e12e3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:38.466904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee82dae-f8d1-4a07-9319-24eb5407805e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c610147005e48a3801c5f5f23067f11, user_id=26fa926d96ee40e69af516af455ecda0, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:38.495608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee82dae-f8d1-4a07-9319-24eb5407805e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:38.495796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c94be614-47d9-47f5-a21d-b88d1e12e3aa None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:38.496949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c94be614-47d9-47f5-a21d-b88d1e12e3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:38.535535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 838/1658] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:30:38.540504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94be614-47d9-47f5-a21d-b88d1e12e3aa None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:38.553165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94be614-47d9-47f5-a21d-b88d1e12e3aa None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:38.558647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 821/1659] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:38 2022] PUT /identity/v3/projects/2c610147005e48a3801c5f5f23067f11/users/26fa926d96ee40e69af516af455ecda0/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:30:38.571973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddbe8e-c3a1-4472-98ea-49a6be8147b3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:38.572263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddbe8e-c3a1-4472-98ea-49a6be8147b3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:38.572544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddbe8e-c3a1-4472-98ea-49a6be8147b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:38.651667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6eddbe8e-c3a1-4472-98ea-49a6be8147b3 None None] MFA Rules not processed for user `26fa926d96ee40e69af516af455ecda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:38.656383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6eddbe8e-c3a1-4472-98ea-49a6be8147b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:38.694835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:38.825013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:38.826209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:38.826553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:38.826847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:38.831073 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:38.831073 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:38.833558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:38.922978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 839/1660] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:38.923970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:38.981420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee74f70-c675-4aaa-afed-612adc18b854 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:39.012781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 822/1661] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:39.014343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddbe8e-c3a1-4472-98ea-49a6be8147b3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:39.018679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:39.024665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:39.024965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:39.025253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:39.025995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:39.042237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:39.074833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddbe8e-c3a1-4472-98ea-49a6be8147b3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:39.101328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb89b-2d76-4f78-a26e-ac75e10aa06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:39.102767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb89b-2d76-4f78-a26e-ac75e10aa06f admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:39.103713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb89b-2d76-4f78-a26e-ac75e10aa06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:39.104363 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb89b-2d76-4f78-a26e-ac75e10aa06f admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:39.106144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fb89b-2d76-4f78-a26e-ac75e10aa06f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:39.112890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fb89b-2d76-4f78-a26e-ac75e10aa06f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:39.178282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 840/1662] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:38 2022] POST /identity/v3/projects => generated 369 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:39.180887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:39.199984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:39.205976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 823/1663] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:39 2022] POST /identity/v3/projects => generated 393 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:39.210100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb89b-2d76-4f78-a26e-ac75e10aa06f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:39.509011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:39.518120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb89b-2d76-4f78-a26e-ac75e10aa06f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:39.535330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f732b2-0830-4efa-8348-1808da121544 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:39.547264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd652-b001-4024-b156-8d3d9c7e67d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:39.548058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd652-b001-4024-b156-8d3d9c7e67d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:39.552868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd652-b001-4024-b156-8d3d9c7e67d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:39.553182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd652-b001-4024-b156-8d3d9c7e67d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:39.554538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbd652-b001-4024-b156-8d3d9c7e67d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:39.563404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbd652-b001-4024-b156-8d3d9c7e67d5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:39.566601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:39.567234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:39.567526 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:39.567811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] PATH_INFO: `/v3/projects/5243c0442fc04fe7b74708c0facdc3ba` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:39.574367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5243c0442fc04fe7b74708c0facdc3ba)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:39.576314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:39.616566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 824/1664] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:39 2022] POST /identity/v3/users => generated 285 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:39.625177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd652-b001-4024-b156-8d3d9c7e67d5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:39.662020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd652-b001-4024-b156-8d3d9c7e67d5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:39.683149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5558c85-3c03-495b-a433-948bb0a74e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:39.685504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5558c85-3c03-495b-a433-948bb0a74e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:39.688166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5558c85-3c03-495b-a433-948bb0a74e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:39.688897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5558c85-3c03-495b-a433-948bb0a74e42 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:39.690009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5558c85-3c03-495b-a433-948bb0a74e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:39.697390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5558c85-3c03-495b-a433-948bb0a74e42 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:39.822104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 825/1665] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:39 2022] POST /identity/v3/users => generated 303 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:39.823107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5558c85-3c03-495b-a433-948bb0a74e42 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:39.836570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5243c0442fc04fe7b74708c0facdc3ba', 'request_id': 'req-8027e106-634c-4b7c-a266-ccf6f931ea57'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:39.853164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5558c85-3c03-495b-a433-948bb0a74e42 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:39.860042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5243c0442fc04fe7b74708c0facdc3ba', 'request_id': 'req-8027e106-634c-4b7c-a266-ccf6f931ea57'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:39.901790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5243c0442fc04fe7b74708c0facdc3ba', 'request_id': 'req-8027e106-634c-4b7c-a266-ccf6f931ea57'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:39.946537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] The token cache is being invalidate because project 5243c0442fc04fe7b74708c0facdc3ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:30:39.946846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:39.955428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:39.960580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 841/1666] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:39 2022] DELETE /identity/v3/projects/5243c0442fc04fe7b74708c0facdc3ba => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:39.960580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:39.962634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:40.045440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:40.067531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6191f4-755b-4872-b2fa-2bedd66213bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:40.069997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6191f4-755b-4872-b2fa-2bedd66213bf admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:40.069997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6191f4-755b-4872-b2fa-2bedd66213bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:40.069997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6191f4-755b-4872-b2fa-2bedd66213bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:40.069997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6191f4-755b-4872-b2fa-2bedd66213bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:40.079658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6191f4-755b-4872-b2fa-2bedd66213bf admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:40.109841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 826/1667] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:40.111345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8d454-7553-46bf-9f1d-8b0df107e7be None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:40.111584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8d454-7553-46bf-9f1d-8b0df107e7be None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:40.111872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8d454-7553-46bf-9f1d-8b0df107e7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:40.216165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027e106-634c-4b7c-a266-ccf6f931ea57 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:40.222809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-dcc8d454-7553-46bf-9f1d-8b0df107e7be None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:40.224088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dcc8d454-7553-46bf-9f1d-8b0df107e7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:40.239825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0211c-28e1-454a-9fd4-4f7e02cb5153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:40.240479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0211c-28e1-454a-9fd4-4f7e02cb5153 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:40.240768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0211c-28e1-454a-9fd4-4f7e02cb5153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:40.241065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0211c-28e1-454a-9fd4-4f7e02cb5153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:40.243983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0211c-28e1-454a-9fd4-4f7e02cb5153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:40.245759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0211c-28e1-454a-9fd4-4f7e02cb5153 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:40.269414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 827/1668] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:40.276812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8d454-7553-46bf-9f1d-8b0df107e7be None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:40.280857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dcc8d454-7553-46bf-9f1d-8b0df107e7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:40.307991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 842/1669] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:40.320533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a472269-46d8-4144-892f-a3f1af455b3e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:40.320704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a472269-46d8-4144-892f-a3f1af455b3e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:40.320981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a472269-46d8-4144-892f-a3f1af455b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:40.376130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8d454-7553-46bf-9f1d-8b0df107e7be None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:40.419116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f69cb6-9862-45e4-84ad-17b5b1368675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:40.423478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f69cb6-9862-45e4-84ad-17b5b1368675 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:40.424070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f69cb6-9862-45e4-84ad-17b5b1368675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:40.424557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f69cb6-9862-45e4-84ad-17b5b1368675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:40.431645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f69cb6-9862-45e4-84ad-17b5b1368675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:40.446362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f69cb6-9862-45e4-84ad-17b5b1368675 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:40.467956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0a472269-46d8-4144-892f-a3f1af455b3e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:40.470675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0a472269-46d8-4144-892f-a3f1af455b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:40.497035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 828/1670] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:40.502853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f69cb6-9862-45e4-84ad-17b5b1368675 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:40.544497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 843/1671] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:40.550647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a472269-46d8-4144-892f-a3f1af455b3e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:40.572019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f69cb6-9862-45e4-84ad-17b5b1368675 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:40.589481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a472269-46d8-4144-892f-a3f1af455b3e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:40.590352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8e873-f192-4e68-b681-499e169823c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:40.616224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8e873-f192-4e68-b681-499e169823c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:40.618867 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8e873-f192-4e68-b681-499e169823c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:40.619336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8e873-f192-4e68-b681-499e169823c3 admin admin] PATH_INFO: `/v3/projects/f34743461ee24f538a9dd84157ecaac5/users/a13a396749404009a1fe181e8290177b/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:40.643347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a2b71-0a88-4f39-bb05-8717f0a55fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:40.645620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2b71-0a88-4f39-bb05-8717f0a55fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:40.648532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2b71-0a88-4f39-bb05-8717f0a55fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:40.650875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2b71-0a88-4f39-bb05-8717f0a55fbf admin admin] PATH_INFO: `/v3/projects/d873eee402274ab1996dbede8a022104/users/427ca0235a034cf28857336b915b2c58/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:40.695668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8e873-f192-4e68-b681-499e169823c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f34743461ee24f538a9dd84157ecaac5, user_id=a13a396749404009a1fe181e8290177b, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:40.698923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8e873-f192-4e68-b681-499e169823c3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:40.719923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a2b71-0a88-4f39-bb05-8717f0a55fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d873eee402274ab1996dbede8a022104, user_id=427ca0235a034cf28857336b915b2c58, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:40.728418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a2b71-0a88-4f39-bb05-8717f0a55fbf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:40.761352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 829/1672] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:40 2022] PUT /identity/v3/projects/f34743461ee24f538a9dd84157ecaac5/users/a13a396749404009a1fe181e8290177b/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:40.772279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8e873-f192-4e68-b681-499e169823c3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:40.806895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 844/1673] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:40 2022] PUT /identity/v3/projects/d873eee402274ab1996dbede8a022104/users/427ca0235a034cf28857336b915b2c58/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:40.808437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a2b71-0a88-4f39-bb05-8717f0a55fbf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:41.099785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8e873-f192-4e68-b681-499e169823c3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:41.106096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:41.132211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:41.133929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:41.134551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:41.134845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a2b71-0a88-4f39-bb05-8717f0a55fbf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:41.135295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:41.148699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:41.148699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:41.152430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21494bf-6a32-4222-9f64-24b3f4082ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:41.152430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21494bf-6a32-4222-9f64-24b3f4082ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:41.152430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21494bf-6a32-4222-9f64-24b3f4082ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:41.152430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21494bf-6a32-4222-9f64-24b3f4082ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:41.153029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21494bf-6a32-4222-9f64-24b3f4082ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:41.165688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21494bf-6a32-4222-9f64-24b3f4082ac0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:41.179016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:41.189626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 845/1674] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:41.191000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a2ee-7b63-43fb-85c4-02edb45ece14 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:41.191297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a2ee-7b63-43fb-85c4-02edb45ece14 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:41.191581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a2ee-7b63-43fb-85c4-02edb45ece14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:41.273225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1b45a2ee-7b63-43fb-85c4-02edb45ece14 None None] MFA Rules not processed for user `427ca0235a034cf28857336b915b2c58`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:41.275033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1b45a2ee-7b63-43fb-85c4-02edb45ece14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:41.288698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 830/1675] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 527 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:41.289670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:41.330088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac3521-024e-4d0e-bfb2-01352c0fb965 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:41.346743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38212f43-df55-4617-bb16-e2227e120e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:41.350044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38212f43-df55-4617-bb16-e2227e120e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:41.350677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38212f43-df55-4617-bb16-e2227e120e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:41.351540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38212f43-df55-4617-bb16-e2227e120e3e admin admin] PATH_INFO: `/v3/projects/f34743461ee24f538a9dd84157ecaac5/users/a13a396749404009a1fe181e8290177b/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:41.360858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38212f43-df55-4617-bb16-e2227e120e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f34743461ee24f538a9dd84157ecaac5, user_id=a13a396749404009a1fe181e8290177b, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:41.373817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38212f43-df55-4617-bb16-e2227e120e3e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:41.466773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 831/1676] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:41 2022] PUT /identity/v3/projects/f34743461ee24f538a9dd84157ecaac5/users/a13a396749404009a1fe181e8290177b/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:41.466773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38212f43-df55-4617-bb16-e2227e120e3e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:41.537708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 846/1677] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:41.545311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31476-b008-4a1a-b10d-4bc7351b723b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:41.554287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31476-b008-4a1a-b10d-4bc7351b723b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:41.554896 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e31476-b008-4a1a-b10d-4bc7351b723b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:41.694723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a5e31476-b008-4a1a-b10d-4bc7351b723b None None] MFA Rules not processed for user `a13a396749404009a1fe181e8290177b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:41.699144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a5e31476-b008-4a1a-b10d-4bc7351b723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:41.728730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38212f43-df55-4617-bb16-e2227e120e3e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:41.741708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe811e33-b88b-4e0c-b162-0a7c7a4a6e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:41.742358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe811e33-b88b-4e0c-b162-0a7c7a4a6e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:41.742639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe811e33-b88b-4e0c-b162-0a7c7a4a6e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:41.742915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe811e33-b88b-4e0c-b162-0a7c7a4a6e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:41.743565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe811e33-b88b-4e0c-b162-0a7c7a4a6e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:41.753905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe811e33-b88b-4e0c-b162-0a7c7a4a6e48 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:41.926498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 832/1678] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:41 2022] POST /identity/v3/projects => generated 365 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:41.928223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe811e33-b88b-4e0c-b162-0a7c7a4a6e48 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:42.063640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 847/1679] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:42.068145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:42.070085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e31476-b008-4a1a-b10d-4bc7351b723b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:42.074464 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a5e31476-b008-4a1a-b10d-4bc7351b723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:42.154492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e31476-b008-4a1a-b10d-4bc7351b723b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:42.210263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe811e33-b88b-4e0c-b162-0a7c7a4a6e48 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:42.228778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97bfc7-efad-4e37-9d2e-344d46fefca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:42.229722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97bfc7-efad-4e37-9d2e-344d46fefca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:42.230159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97bfc7-efad-4e37-9d2e-344d46fefca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:42.230709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97bfc7-efad-4e37-9d2e-344d46fefca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:42.231605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97bfc7-efad-4e37-9d2e-344d46fefca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:42.234901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97bfc7-efad-4e37-9d2e-344d46fefca7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:42.304444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 833/1680] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:41 2022] POST /identity/v3/projects => generated 369 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:42.305832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97bfc7-efad-4e37-9d2e-344d46fefca7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:42.348054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:42.348743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:42.349034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:42.349365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] PATH_INFO: `/v3/users/ddbe4e9452c241bfbeddbdb5e51a865b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:42.359104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddbe4e9452c241bfbeddbdb5e51a865b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:42.362763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:42.370408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:42.402557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97bfc7-efad-4e37-9d2e-344d46fefca7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:42.430336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:42.431835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:42.432181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:42.432644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:42.433672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:42.436032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:42.496152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 834/1681] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:42 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:42.498837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:42.504478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:42.555327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddbe4e9452c241bfbeddbdb5e51a865b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:42.591514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddbe4e9452c241bfbeddbdb5e51a865b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:42.594494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddbe4e9452c241bfbeddbdb5e51a865b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:42.609059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddbe4e9452c241bfbeddbdb5e51a865b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:42.628019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddbe4e9452c241bfbeddbdb5e51a865b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:42.640788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 848/1682] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:42 2022] DELETE /identity/v3/users/ddbe4e9452c241bfbeddbdb5e51a865b => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:42.642262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:42.646193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:42.705719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b31c-8d0a-4099-9731-932db2d0250a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:42.874295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:42.879581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2251d6-f4cf-415c-b638-58d341c6395a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:42.914641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:42.914838 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a0bbc-f333-4230-94bd-e094beea9c7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:42.915902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0bbc-f333-4230-94bd-e094beea9c7a service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:42.917597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0bbc-f333-4230-94bd-e094beea9c7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:42.919580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0bbc-f333-4230-94bd-e094beea9c7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:42.923378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3c3a0bbc-f333-4230-94bd-e094beea9c7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:42.926143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:42.926349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:42.927922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:42.927922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:42.932846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:43.004343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 849/1683] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:42 2022] POST /identity/v3/users => generated 291 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:43.005216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:43.020209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:43.047517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37403b03-1a06-4267-8a0b-4baa0ac4387a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:43.067170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060ed1a-ba56-480d-a8c4-9577a4207ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:43.067808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060ed1a-ba56-480d-a8c4-9577a4207ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:43.068119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060ed1a-ba56-480d-a8c4-9577a4207ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:43.068502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060ed1a-ba56-480d-a8c4-9577a4207ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:43.069157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060ed1a-ba56-480d-a8c4-9577a4207ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:43.077485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060ed1a-ba56-480d-a8c4-9577a4207ab4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:43.094078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a0bbc-f333-4230-94bd-e094beea9c7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:43.129418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a0bbc-f333-4230-94bd-e094beea9c7a service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:43.160971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 835/1684] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:43.181257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e685-15f9-4bfa-81d3-c0eb59d1c68e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:43.181630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e685-15f9-4bfa-81d3-c0eb59d1c68e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:43.181914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481e685-15f9-4bfa-81d3-c0eb59d1c68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:43.184483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 850/1685] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:43.185211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060ed1a-ba56-480d-a8c4-9577a4207ab4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:43.243972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060ed1a-ba56-480d-a8c4-9577a4207ab4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:43.271741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8539f-0f9c-467b-a150-60e2c9981028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:43.272899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8539f-0f9c-467b-a150-60e2c9981028 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:43.273191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8539f-0f9c-467b-a150-60e2c9981028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:43.273491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8539f-0f9c-467b-a150-60e2c9981028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:43.274186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8539f-0f9c-467b-a150-60e2c9981028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:43.276272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8539f-0f9c-467b-a150-60e2c9981028 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:43.323945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 851/1686] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:43.329028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3481e685-15f9-4bfa-81d3-c0eb59d1c68e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:43.333507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8539f-0f9c-467b-a150-60e2c9981028 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:43.345535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3481e685-15f9-4bfa-81d3-c0eb59d1c68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:43.372995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8539f-0f9c-467b-a150-60e2c9981028 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:43.403601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:43.404454 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:43.404884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:43.407319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 836/1687] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:43.407948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:43.408788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:43.412673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:43.415157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481e685-15f9-4bfa-81d3-c0eb59d1c68e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:43.470760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481e685-15f9-4bfa-81d3-c0eb59d1c68e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:43.485991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 852/1688] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:43.487023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:43.506374 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625ccca-29e9-454d-8362-0f1bc4f99c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:43.508704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625ccca-29e9-454d-8362-0f1bc4f99c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:43.508704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625ccca-29e9-454d-8362-0f1bc4f99c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:43.508704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625ccca-29e9-454d-8362-0f1bc4f99c24 admin admin] PATH_INFO: `/v3/projects/199c3cb5f1d843228356cbf4983158bb/users/e4a07d3ace804951840c33c6bbbd9ec0/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:43.516760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:43.585607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625ccca-29e9-454d-8362-0f1bc4f99c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199c3cb5f1d843228356cbf4983158bb, user_id=e4a07d3ace804951840c33c6bbbd9ec0, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:43.588980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625ccca-29e9-454d-8362-0f1bc4f99c24 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:43.609499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808393-4133-4a74-9cf7-04ee5eebc5f4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:43.620573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:43.621499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:43.622365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:43.622848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:43.630419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:43.672819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 837/1689] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:43 2022] PUT /identity/v3/projects/199c3cb5f1d843228356cbf4983158bb/users/e4a07d3ace804951840c33c6bbbd9ec0/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:43.680344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625ccca-29e9-454d-8362-0f1bc4f99c24 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:43.863190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:43.961257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:43.965823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:44.019175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 853/1690] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:44.019997 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:44.032927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625ccca-29e9-454d-8362-0f1bc4f99c24 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:44.034548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:44.050218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907ded8-56e4-4f00-811d-bbc60bf1fb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:44.050936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ded8-56e4-4f00-811d-bbc60bf1fb69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:44.051223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ded8-56e4-4f00-811d-bbc60bf1fb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:44.051505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ded8-56e4-4f00-811d-bbc60bf1fb69 admin admin] PATH_INFO: `/v3/projects/33d7dd0b35db44d98d484f13e733a143/users/a57413c042d444d592d8f21b35c6da28/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:44.160609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907ded8-56e4-4f00-811d-bbc60bf1fb69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d7dd0b35db44d98d484f13e733a143, user_id=a57413c042d444d592d8f21b35c6da28, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:44.165165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907ded8-56e4-4f00-811d-bbc60bf1fb69 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:44.199470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 838/1691] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:43 2022] PUT /identity/v3/projects/33d7dd0b35db44d98d484f13e733a143/users/a57413c042d444d592d8f21b35c6da28/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:44.210839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907ded8-56e4-4f00-811d-bbc60bf1fb69 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:44.270185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16f535-7ce6-47ed-84d0-2fcf0709b31d service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:44.360507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:44.382648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907ded8-56e4-4f00-811d-bbc60bf1fb69 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:44.390915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4fbe2-81ad-48a4-b8b7-b3da32074a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:44.391559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4fbe2-81ad-48a4-b8b7-b3da32074a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:44.392320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4fbe2-81ad-48a4-b8b7-b3da32074a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:44.393894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4fbe2-81ad-48a4-b8b7-b3da32074a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:44.394251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4fbe2-81ad-48a4-b8b7-b3da32074a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:44.395899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4fbe2-81ad-48a4-b8b7-b3da32074a39 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:44.418003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 839/1692] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:44.419530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7edce-6fff-491a-ade8-f98c75e9ef7a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:44.419829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7edce-6fff-491a-ade8-f98c75e9ef7a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:44.420090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e7edce-6fff-491a-ade8-f98c75e9ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:44.476514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-29e7edce-6fff-491a-ade8-f98c75e9ef7a None None] MFA Rules not processed for user `a57413c042d444d592d8f21b35c6da28`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:44.477956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-29e7edce-6fff-491a-ade8-f98c75e9ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:44.531771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ea8ae-e47a-4044-b073-02e037bf2750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:44.532477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ea8ae-e47a-4044-b073-02e037bf2750 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:44.532771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ea8ae-e47a-4044-b073-02e037bf2750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:44.533056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ea8ae-e47a-4044-b073-02e037bf2750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:44.603104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ea8ae-e47a-4044-b073-02e037bf2750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:44.616587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ea8ae-e47a-4044-b073-02e037bf2750 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:44.675925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 854/1693] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:44.678550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ea8ae-e47a-4044-b073-02e037bf2750 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:44.760381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 840/1694] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:44.761498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7edce-6fff-491a-ade8-f98c75e9ef7a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:44.767742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ea8ae-e47a-4044-b073-02e037bf2750 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:44.786107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e7edce-6fff-491a-ade8-f98c75e9ef7a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:44.805711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030378d-a74b-4b76-a030-ac1d67061bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:44.807319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030378d-a74b-4b76-a030-ac1d67061bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:44.807616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030378d-a74b-4b76-a030-ac1d67061bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:44.810217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a41a5-ccfe-4784-b6d6-2ba7e234405f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:44.810941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a41a5-ccfe-4784-b6d6-2ba7e234405f admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:44.811298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a41a5-ccfe-4784-b6d6-2ba7e234405f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:44.811727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a41a5-ccfe-4784-b6d6-2ba7e234405f admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:44.812587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a41a5-ccfe-4784-b6d6-2ba7e234405f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:44.814706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a41a5-ccfe-4784-b6d6-2ba7e234405f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:44.815954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030378d-a74b-4b76-a030-ac1d67061bb6 admin admin] PATH_INFO: `/v3/projects/199c3cb5f1d843228356cbf4983158bb/users/e4a07d3ace804951840c33c6bbbd9ec0/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:44.821996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030378d-a74b-4b76-a030-ac1d67061bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199c3cb5f1d843228356cbf4983158bb, user_id=e4a07d3ace804951840c33c6bbbd9ec0, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:44.847562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5030378d-a74b-4b76-a030-ac1d67061bb6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:44.892298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 841/1695] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:44 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:30:44.898269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a41a5-ccfe-4784-b6d6-2ba7e234405f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:44.908497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 855/1696] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:44 2022] PUT /identity/v3/projects/199c3cb5f1d843228356cbf4983158bb/users/e4a07d3ace804951840c33c6bbbd9ec0/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:44.920239 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba9e7c-a30b-4adc-867d-1e65fd103ab8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:44.920521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba9e7c-a30b-4adc-867d-1e65fd103ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:44.920801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba9e7c-a30b-4adc-867d-1e65fd103ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:45.016106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-88ba9e7c-a30b-4adc-867d-1e65fd103ab8 None None] MFA Rules not processed for user `e4a07d3ace804951840c33c6bbbd9ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:45.017248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-88ba9e7c-a30b-4adc-867d-1e65fd103ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:45.142923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a41a5-ccfe-4784-b6d6-2ba7e234405f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:45.167387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:45.168315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:45.168693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:45.168941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:45.171059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:45.173577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:45.226717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 842/1697] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:44 2022] POST /identity/v3/users => generated 288 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:45.226717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:45.231202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:45.282305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 856/1698] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:45.297633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b00fd-99bf-4207-9a79-c1df509aa8be admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:45.299188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba9e7c-a30b-4adc-867d-1e65fd103ab8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:45.316468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:45.320974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:45.321842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:45.323940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] PATH_INFO: `/v3/users/e7ecab5bc21049e9991d63a79779b7d2` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:45.329372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ecab5bc21049e9991d63a79779b7d2)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:45.335298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:45.339795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba9e7c-a30b-4adc-867d-1e65fd103ab8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:45.356855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:45.358603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:45.360520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:45.361028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:45.365868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:45.370983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:45.378497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:45.416215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 857/1699] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:45.418812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:45.423193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:45.467597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:45.484229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ecab5bc21049e9991d63a79779b7d2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:45.485216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ecab5bc21049e9991d63a79779b7d2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:45.486211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ecab5bc21049e9991d63a79779b7d2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:45.505878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ecab5bc21049e9991d63a79779b7d2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:45.528636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ecab5bc21049e9991d63a79779b7d2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:45.563571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 843/1700] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:45 2022] DELETE /identity/v3/users/e7ecab5bc21049e9991d63a79779b7d2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:45.564668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:45.570018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:45.607758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e08-9f95-4617-a6d4-2ac2e1e3216e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:45.788142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:45.788858 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:45.789226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:45.789617 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:45.808166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:45.849584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8cdb7-161e-4d36-9f8a-855bd194f258 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:45.868120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e196e-327c-42bb-a29b-d5b96830f25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:45.869239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e196e-327c-42bb-a29b-d5b96830f25b admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:45.869778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e196e-327c-42bb-a29b-d5b96830f25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:45.870502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e196e-327c-42bb-a29b-d5b96830f25b admin admin] PATH_INFO: `/v3/projects/74e7a503942c4b3d8ab7955f98b2d794/users/51e75998702847e2a39aca23c1ca53bd/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:45.914601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e196e-327c-42bb-a29b-d5b96830f25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e7a503942c4b3d8ab7955f98b2d794, user_id=51e75998702847e2a39aca23c1ca53bd, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:45.920031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e196e-327c-42bb-a29b-d5b96830f25b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:45.970649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 844/1701] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:45 2022] PUT /identity/v3/projects/74e7a503942c4b3d8ab7955f98b2d794/users/51e75998702847e2a39aca23c1ca53bd/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:45.979123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d4ac-3d40-40ff-946c-11ea58a098b1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:45.981111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d4ac-3d40-40ff-946c-11ea58a098b1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:45.981572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d4ac-3d40-40ff-946c-11ea58a098b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:46.039202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:46.041752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:46.043232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c129d4ac-3d40-40ff-946c-11ea58a098b1 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:46.046577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c129d4ac-3d40-40ff-946c-11ea58a098b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:46.079778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 858/1702] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:46.080792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:46.107191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65393d1-93ca-45f8-a73a-5fa7ec28b30a service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:46.323337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 845/1703] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:46.324844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4270df2-29b8-43cc-b5e1-0330c5dc20ba None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:46.327533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4270df2-29b8-43cc-b5e1-0330c5dc20ba None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:46.327533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4270df2-29b8-43cc-b5e1-0330c5dc20ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:46.445475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:46.447836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:46.448541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:46.448972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:46.455285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:46.457249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:46.469966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:46.472515 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b4270df2-29b8-43cc-b5e1-0330c5dc20ba None None] MFA Rules not processed for user `b8defdf430274affaed766d8aedccfad`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:46.474829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b4270df2-29b8-43cc-b5e1-0330c5dc20ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:46.504188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 859/1704] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:46.507661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:46.516312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:46.558686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:46.766270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee130d8-72a7-4a51-a667-41ccb644e85b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:46.778866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:46.786653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:46.790275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:46.790275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:46.802342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:46.821193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 846/1705] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:46.822354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4270df2-29b8-43cc-b5e1-0330c5dc20ba None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:46.827225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b4270df2-29b8-43cc-b5e1-0330c5dc20ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:47.013463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:47.019954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:47.046409 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 860/1706] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:47.047485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:47.074724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88804f9-bc28-4654-b7f3-970c4cabba35 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:47.088320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4270df2-29b8-43cc-b5e1-0330c5dc20ba None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:47.095041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ad11c-6a93-4541-a8ff-ce3a560a42bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:47.095798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ad11c-6a93-4541-a8ff-ce3a560a42bf service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:47.102432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2217d0-97f1-49fb-b735-01565526ff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:47.105411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2217d0-97f1-49fb-b735-01565526ff91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:47.107048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2217d0-97f1-49fb-b735-01565526ff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:47.107444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2217d0-97f1-49fb-b735-01565526ff91 admin admin] PATH_INFO: `/v3/projects/74e7a503942c4b3d8ab7955f98b2d794/users/51e75998702847e2a39aca23c1ca53bd/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:47.109214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ad11c-6a93-4541-a8ff-ce3a560a42bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:47.110196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ad11c-6a93-4541-a8ff-ce3a560a42bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:47.123954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2217d0-97f1-49fb-b735-01565526ff91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e7a503942c4b3d8ab7955f98b2d794, user_id=51e75998702847e2a39aca23c1ca53bd, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:47.128516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2217d0-97f1-49fb-b735-01565526ff91 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:47.155362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ad11c-6a93-4541-a8ff-ce3a560a42bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:47.160150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ad11c-6a93-4541-a8ff-ce3a560a42bf service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:47.189043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 861/1707] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:47 2022] PUT /identity/v3/projects/74e7a503942c4b3d8ab7955f98b2d794/users/51e75998702847e2a39aca23c1ca53bd/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:47.193325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29bb5e-a8aa-4e5f-9d51-c4b3e3bb9ddc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:47.194174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29bb5e-a8aa-4e5f-9d51-c4b3e3bb9ddc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:47.197832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29bb5e-a8aa-4e5f-9d51-c4b3e3bb9ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:47.237191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 847/1708] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:47.239631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122b15-c799-48f1-9b77-eed8e0d3e28c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:47.254049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122b15-c799-48f1-9b77-eed8e0d3e28c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:47.254049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122b15-c799-48f1-9b77-eed8e0d3e28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:47.308038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2e29bb5e-a8aa-4e5f-9d51-c4b3e3bb9ddc None None] MFA Rules not processed for user `b8defdf430274affaed766d8aedccfad`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:47.309497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2e29bb5e-a8aa-4e5f-9d51-c4b3e3bb9ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:47.482412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-80122b15-c799-48f1-9b77-eed8e0d3e28c None None] MFA Rules not processed for user `51e75998702847e2a39aca23c1ca53bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:47.492928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-80122b15-c799-48f1-9b77-eed8e0d3e28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:47.662646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 862/1709] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:47.669342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839af19-a793-4a08-80bd-15aa6e4fc4ba None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:47.673975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839af19-a793-4a08-80bd-15aa6e4fc4ba None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:47.676208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839af19-a793-4a08-80bd-15aa6e4fc4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:47.896984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 848/1710] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 651 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:47.898342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80122b15-c799-48f1-9b77-eed8e0d3e28c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:47.907186 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-80122b15-c799-48f1-9b77-eed8e0d3e28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:47.951023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4839af19-a793-4a08-80bd-15aa6e4fc4ba None None] MFA Rules not processed for user `98de1ccbabc64162a29cbce2cfe11600`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:47.954187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4839af19-a793-4a08-80bd-15aa6e4fc4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:48.124911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:48.145832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:48.248608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80122b15-c799-48f1-9b77-eed8e0d3e28c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:48.289700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3771e06-1a4f-4917-bd86-fcfb998404fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:48.293609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3771e06-1a4f-4917-bd86-fcfb998404fe service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:48.293933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3771e06-1a4f-4917-bd86-fcfb998404fe service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:48.294212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3771e06-1a4f-4917-bd86-fcfb998404fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:48.316401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 863/1711] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 653 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:48.317704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839af19-a793-4a08-80bd-15aa6e4fc4ba None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:48.332784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4839af19-a793-4a08-80bd-15aa6e4fc4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:48.340908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3771e06-1a4f-4917-bd86-fcfb998404fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:48.348381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3771e06-1a4f-4917-bd86-fcfb998404fe service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:48.415576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 849/1712] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:48.416517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3771e06-1a4f-4917-bd86-fcfb998404fe service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:48.650422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839af19-a793-4a08-80bd-15aa6e4fc4ba None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:48.657017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:48.658730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:48.659315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:48.659744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] PATH_INFO: `/v3/projects/277f039d5372403cae3f766dfd01c81c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:48.661505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277f039d5372403cae3f766dfd01c81c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:48.675551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:48.695781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3771e06-1a4f-4917-bd86-fcfb998404fe service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:48.724163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:48.725263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:48.725857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:48.726341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:48.775878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:48.783861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:48.834655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277f039d5372403cae3f766dfd01c81c', 'request_id': 'req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:48.850111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 850/1713] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:48.852327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:48.853946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277f039d5372403cae3f766dfd01c81c', 'request_id': 'req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:48.858289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:48.893428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277f039d5372403cae3f766dfd01c81c', 'request_id': 'req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:48.927308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] The token cache is being invalidate because project 277f039d5372403cae3f766dfd01c81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:30:48.928121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:48.931631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13407f81-f6c9-4c68-8ba0-c3e2ee64f455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:48.935276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf97ec-b1df-4c0f-bad4-150a57c97307 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:48.935471 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 864/1714] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:48 2022] DELETE /identity/v3/projects/277f039d5372403cae3f766dfd01c81c => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:48.950967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec149f8-ca63-4b70-9ab4-d1f6cc08f839 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:48.951445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec149f8-ca63-4b70-9ab4-d1f6cc08f839 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:48.951947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec149f8-ca63-4b70-9ab4-d1f6cc08f839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:49.053169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-eec149f8-ca63-4b70-9ab4-d1f6cc08f839 None None] MFA Rules not processed for user `ae71fd7ba4334d0d813cbc0c06e3686e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:49.055931 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-eec149f8-ca63-4b70-9ab4-d1f6cc08f839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:49.232003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5acb34-2600-4fb7-9229-c1179d1e3f2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:49.232927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5acb34-2600-4fb7-9229-c1179d1e3f2b service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:49.232927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5acb34-2600-4fb7-9229-c1179d1e3f2b service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:49.233798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5acb34-2600-4fb7-9229-c1179d1e3f2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:49.241832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1d5acb34-2600-4fb7-9229-c1179d1e3f2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:49.271904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:49.307469 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 865/1715] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:49.308520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec149f8-ca63-4b70-9ab4-d1f6cc08f839 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:49.316903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-eec149f8-ca63-4b70-9ab4-d1f6cc08f839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:49.387678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5acb34-2600-4fb7-9229-c1179d1e3f2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:49.389722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5acb34-2600-4fb7-9229-c1179d1e3f2b service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:49.402234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 851/1716] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:49.404191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c9c39-f07f-4f02-9384-a74bfc7e0f12 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:49.404641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c9c39-f07f-4f02-9384-a74bfc7e0f12 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:49.405239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c9c39-f07f-4f02-9384-a74bfc7e0f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:49.432270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-732c9c39-f07f-4f02-9384-a74bfc7e0f12 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:49.434083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-732c9c39-f07f-4f02-9384-a74bfc7e0f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:49.478447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:49.619990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 852/1717] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:49.620981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c9c39-f07f-4f02-9384-a74bfc7e0f12 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:49.629012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-732c9c39-f07f-4f02-9384-a74bfc7e0f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:49.671703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec149f8-ca63-4b70-9ab4-d1f6cc08f839 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:49.683183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:49.685424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:49.686171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:49.687813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] PATH_INFO: `/v3/projects/f402f6d4d8964fa1a5c0275873535fa1` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:49.690588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f402f6d4d8964fa1a5c0275873535fa1)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:49.694450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:49.820688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c9c39-f07f-4f02-9384-a74bfc7e0f12 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:49.871708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f402f6d4d8964fa1a5c0275873535fa1', 'request_id': 'req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:49.893491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f402f6d4d8964fa1a5c0275873535fa1', 'request_id': 'req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:49.910658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f402f6d4d8964fa1a5c0275873535fa1', 'request_id': 'req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:49.970265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f3168-0486-49e3-964d-dcf8b4b30f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:49.971048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f3168-0486-49e3-964d-dcf8b4b30f76 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:49.971640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f3168-0486-49e3-964d-dcf8b4b30f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:49.972325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f3168-0486-49e3-964d-dcf8b4b30f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:49.977506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-546f3168-0486-49e3-964d-dcf8b4b30f76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:49.978814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] The token cache is being invalidate because project f402f6d4d8964fa1a5c0275873535fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:30:49.979137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:49.986429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3ee6b251-54eb-4b6a-a91d-a2a79bd1ef37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:49.997335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 866/1718] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:49 2022] DELETE /identity/v3/projects/f402f6d4d8964fa1a5c0275873535fa1 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:50.004234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871d58e-e804-4c09-9c8c-c022f48d0205 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:50.004530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871d58e-e804-4c09-9c8c-c022f48d0205 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:50.004814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871d58e-e804-4c09-9c8c-c022f48d0205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:50.053433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b871d58e-e804-4c09-9c8c-c022f48d0205 None None] MFA Rules not processed for user `ae71fd7ba4334d0d813cbc0c06e3686e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:50.055773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b871d58e-e804-4c09-9c8c-c022f48d0205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:50.156341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 867/1719] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:50.161346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f9e0-d3b2-4080-997f-f3bac4494c62 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:50.161346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f9e0-d3b2-4080-997f-f3bac4494c62 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:50.161346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030f9e0-d3b2-4080-997f-f3bac4494c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:50.195096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a030f9e0-d3b2-4080-997f-f3bac4494c62 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:50.197200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a030f9e0-d3b2-4080-997f-f3bac4494c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:50.245586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:50.286966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f3168-0486-49e3-964d-dcf8b4b30f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:50.291021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f3168-0486-49e3-964d-dcf8b4b30f76 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:50.322066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 853/1720] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:50.327378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abd8d1-b26d-4b67-ab08-cfbf95a7b8cd None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:50.329049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abd8d1-b26d-4b67-ab08-cfbf95a7b8cd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:50.331314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abd8d1-b26d-4b67-ab08-cfbf95a7b8cd None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:50.336670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 854/1721] 10.222.0.58 () {54 vars in 912 bytes} [Mon Oct 3 13:30:50 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:30:50.347523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558752de-805c-4f70-a938-2ec9af1a264b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:50.347765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558752de-805c-4f70-a938-2ec9af1a264b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:50.348029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558752de-805c-4f70-a938-2ec9af1a264b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:50.353887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 868/1722] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:50.361173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f9e0-d3b2-4080-997f-f3bac4494c62 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:50.363920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a030f9e0-d3b2-4080-997f-f3bac4494c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:50.421614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-558752de-805c-4f70-a938-2ec9af1a264b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:50.428777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-558752de-805c-4f70-a938-2ec9af1a264b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:50.449157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 855/1723] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:50.454146 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42383f45-3491-4f80-81d4-74096a81a1d0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:50.454460 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42383f45-3491-4f80-81d4-74096a81a1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:50.454702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42383f45-3491-4f80-81d4-74096a81a1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:50.596909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-42383f45-3491-4f80-81d4-74096a81a1d0 None None] MFA Rules not processed for user `cf865cb3c7fc432884e4e308c397c83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:50.602906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-42383f45-3491-4f80-81d4-74096a81a1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:50.643219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:50.659176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030f9e0-d3b2-4080-997f-f3bac4494c62 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:50.676494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:50.678891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:50.680213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:50.680959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:50.685143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:50.731408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:50.744831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:50.826223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 869/1724] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:50.829618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:50.852090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc811c63-94bf-4fc8-9a7f-6c15ec9ea934 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:50.862026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bebc0-4ce4-465d-b3d1-0a110a258990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:50.863072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bebc0-4ce4-465d-b3d1-0a110a258990 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:50.863554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bebc0-4ce4-465d-b3d1-0a110a258990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:50.864033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bebc0-4ce4-465d-b3d1-0a110a258990 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:50.865008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bebc0-4ce4-465d-b3d1-0a110a258990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:50.865573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bebc0-4ce4-465d-b3d1-0a110a258990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:50.869772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bebc0-4ce4-465d-b3d1-0a110a258990 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:50.913801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 870/1725] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:50.913801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36222938-7cbe-44c4-aa0c-153545a0a633 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:50.914471 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36222938-7cbe-44c4-aa0c-153545a0a633 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:50.914943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 856/1726] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:30:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:50.915433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36222938-7cbe-44c4-aa0c-153545a0a633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:50.919312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42383f45-3491-4f80-81d4-74096a81a1d0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:50.991141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-36222938-7cbe-44c4-aa0c-153545a0a633 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:51.004430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-36222938-7cbe-44c4-aa0c-153545a0a633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:51.012042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42383f45-3491-4f80-81d4-74096a81a1d0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.025416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 871/1727] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:51.025416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36222938-7cbe-44c4-aa0c-153545a0a633 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.025416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-36222938-7cbe-44c4-aa0c-153545a0a633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:51.031983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fc6a2-d3e2-4bf5-879b-a77150f9c98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.032258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fc6a2-d3e2-4bf5-879b-a77150f9c98b admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.032632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fc6a2-d3e2-4bf5-879b-a77150f9c98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.032782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912fc6a2-d3e2-4bf5-879b-a77150f9c98b admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.033433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fc6a2-d3e2-4bf5-879b-a77150f9c98b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.035653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912fc6a2-d3e2-4bf5-879b-a77150f9c98b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.044858 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36222938-7cbe-44c4-aa0c-153545a0a633 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.057541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2ad5a-603f-4453-93bb-14242ccee23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.058201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2ad5a-603f-4453-93bb-14242ccee23f service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.058500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2ad5a-603f-4453-93bb-14242ccee23f service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.058857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2ad5a-603f-4453-93bb-14242ccee23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.082206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2ad5a-603f-4453-93bb-14242ccee23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.084341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2ad5a-603f-4453-93bb-14242ccee23f service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.089430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 857/1728] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:50 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:51.090332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fc6a2-d3e2-4bf5-879b-a77150f9c98b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.121778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 872/1729] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:51.127134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2ad5a-603f-4453-93bb-14242ccee23f service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.170595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2ad5a-603f-4453-93bb-14242ccee23f service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.227648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912fc6a2-d3e2-4bf5-879b-a77150f9c98b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.239152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.239801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.240072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.240347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.241015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:30:51.241478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.243303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.278623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 858/1730] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:30:51.286093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.301324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a4047-9bac-49e2-991a-40b116b13dcb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.309533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f30f2-315b-42a8-af2d-0c662ae02c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.310757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f30f2-315b-42a8-af2d-0c662ae02c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.311488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f30f2-315b-42a8-af2d-0c662ae02c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.312282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f30f2-315b-42a8-af2d-0c662ae02c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.314560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f30f2-315b-42a8-af2d-0c662ae02c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.319928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f30f2-315b-42a8-af2d-0c662ae02c6a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.346164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.346547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.349524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.349524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.349524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.350165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.384751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 873/1731] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:51 2022] POST /identity/v3/users => generated 287 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:51.391701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.393972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:51.415929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 859/1732] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:51 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:30:51.417818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f30f2-315b-42a8-af2d-0c662ae02c6a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.423900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4eb79-cbda-4a3f-aafa-384c16a2842f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.445216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:51.639252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.641238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.642600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.643765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] PATH_INFO: `/v3/users/26fa926d96ee40e69af516af455ecda0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.651442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26fa926d96ee40e69af516af455ecda0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.657794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.694039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f30f2-315b-42a8-af2d-0c662ae02c6a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.710488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f04d9f-cc26-4e90-9ef4-b7a95d2a6c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.711406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f04d9f-cc26-4e90-9ef4-b7a95d2a6c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.711988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f04d9f-cc26-4e90-9ef4-b7a95d2a6c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.713536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f04d9f-cc26-4e90-9ef4-b7a95d2a6c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.720817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f04d9f-cc26-4e90-9ef4-b7a95d2a6c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.722872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f04d9f-cc26-4e90-9ef4-b7a95d2a6c54 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.757605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 860/1733] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:51.758881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f04d9f-cc26-4e90-9ef4-b7a95d2a6c54 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.785343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f04d9f-cc26-4e90-9ef4-b7a95d2a6c54 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.801900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26fa926d96ee40e69af516af455ecda0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:51.806325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cde2e-e01f-4932-a888-518c9ed08566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.806976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cde2e-e01f-4932-a888-518c9ed08566 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.807766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cde2e-e01f-4932-a888-518c9ed08566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.808095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7cde2e-e01f-4932-a888-518c9ed08566 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.808852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7cde2e-e01f-4932-a888-518c9ed08566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.810993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7cde2e-e01f-4932-a888-518c9ed08566 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.823017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:51.845882 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 861/1734] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:51 2022] POST /identity/v3/users => generated 291 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:51.847041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cde2e-e01f-4932-a888-518c9ed08566 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.853723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26fa926d96ee40e69af516af455ecda0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:51.862004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26fa926d96ee40e69af516af455ecda0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:51.887373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26fa926d96ee40e69af516af455ecda0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:51.898347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26fa926d96ee40e69af516af455ecda0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:51.902931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7cde2e-e01f-4932-a888-518c9ed08566 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:51.920579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1217f-0e25-46d1-95ae-2b56edb24a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:51.922529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 874/1735] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:51 2022] DELETE /identity/v3/users/26fa926d96ee40e69af516af455ecda0 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:51.923745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1217f-0e25-46d1-95ae-2b56edb24a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.923869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:51.924302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1217f-0e25-46d1-95ae-2b56edb24a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.924819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1217f-0e25-46d1-95ae-2b56edb24a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:51.926137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1217f-0e25-46d1-95ae-2b56edb24a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:51.927213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:51.937612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1217f-0e25-46d1-95ae-2b56edb24a26 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:51.979834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 862/1736] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:51.980756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5061-eb6b-4563-be67-2eb0683390e5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:51.981988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5061-eb6b-4563-be67-2eb0683390e5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:51.982665 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5061-eb6b-4563-be67-2eb0683390e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:52.079831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ff7e5061-eb6b-4563-be67-2eb0683390e5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:52.081983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ff7e5061-eb6b-4563-be67-2eb0683390e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:52.110939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd193564-92df-48b4-9ea2-5cbc84de99e5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:52.123459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:52.124735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:52.125362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:52.125767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:52.126564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:52.130270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 863/1737] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:52.139738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:52.156013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e5061-eb6b-4563-be67-2eb0683390e5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:52.172186 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ff7e5061-eb6b-4563-be67-2eb0683390e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:52.184221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 875/1738] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:52.184221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:52.190627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:52.223447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e5061-eb6b-4563-be67-2eb0683390e5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:52.244603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fb834-c791-444b-b5f8-e140db17078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:52.245330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fb834-c791-444b-b5f8-e140db17078c admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:52.245661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fb834-c791-444b-b5f8-e140db17078c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:52.245943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fb834-c791-444b-b5f8-e140db17078c admin admin] PATH_INFO: `/v3/projects/c9a23b3336e84fc5a490e2a07fed6c13/users/2f2b7f027b5f4d7692ba3ad4d9593c18/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:52.301519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:52.316068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fb834-c791-444b-b5f8-e140db17078c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a23b3336e84fc5a490e2a07fed6c13, user_id=2f2b7f027b5f4d7692ba3ad4d9593c18, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:52.324848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fb834-c791-444b-b5f8-e140db17078c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:52.380997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 864/1739] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:52 2022] PUT /identity/v3/projects/c9a23b3336e84fc5a490e2a07fed6c13/users/2f2b7f027b5f4d7692ba3ad4d9593c18/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:52.382338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fb834-c791-444b-b5f8-e140db17078c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:52.441182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b6514-692f-4aa9-bc92-8a3bfd3b8436 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:52.624592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fb834-c791-444b-b5f8-e140db17078c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:52.646108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7c038-97d1-4d92-b755-68df68881ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:52.648843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c038-97d1-4d92-b755-68df68881ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:52.651074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c038-97d1-4d92-b755-68df68881ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:52.651942 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c038-97d1-4d92-b755-68df68881ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:52.656443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7c038-97d1-4d92-b755-68df68881ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:52.669532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7c038-97d1-4d92-b755-68df68881ae4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:52.697517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:52.698155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:52.698436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:52.698715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:52.735020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:52.736570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:52.749956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 865/1740] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:52.752550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba8f5e-17c1-4922-b793-4dcc82691374 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:52.753810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba8f5e-17c1-4922-b793-4dcc82691374 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:52.753810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba8f5e-17c1-4922-b793-4dcc82691374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:52.767774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 876/1741] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:52.768438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:52.778786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:52.868253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e5ba8f5e-17c1-4922-b793-4dcc82691374 None None] MFA Rules not processed for user `2f2b7f027b5f4d7692ba3ad4d9593c18`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:52.871886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e5ba8f5e-17c1-4922-b793-4dcc82691374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:53.028052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63afe72-d6dd-43f6-ad94-8f03153cd9e6 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:53.056549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7d89f-c71f-4dc5-971f-267a29adce84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:53.057210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d89f-c71f-4dc5-971f-267a29adce84 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:53.057541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d89f-c71f-4dc5-971f-267a29adce84 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:53.057808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d89f-c71f-4dc5-971f-267a29adce84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:53.070441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0fa7d89f-c71f-4dc5-971f-267a29adce84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:53.106437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7d89f-c71f-4dc5-971f-267a29adce84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:53.108369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7d89f-c71f-4dc5-971f-267a29adce84 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:53.126550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 866/1742] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:53.132554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba8f5e-17c1-4922-b793-4dcc82691374 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:53.145118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 877/1743] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:53.146591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89774e2c-83cb-47a0-a16b-b0135d839216 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:53.146895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89774e2c-83cb-47a0-a16b-b0135d839216 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:53.147147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89774e2c-83cb-47a0-a16b-b0135d839216 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:53.153126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 878/1744] 10.222.0.58 () {54 vars in 912 bytes} [Mon Oct 3 13:30:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:30:53.154076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89774e2c-83cb-47a0-a16b-b0135d839216 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:53.162699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba8f5e-17c1-4922-b793-4dcc82691374 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:53.190132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc59a79-876e-4f91-9821-b384e1169892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:53.191539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc59a79-876e-4f91-9821-b384e1169892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:53.192044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc59a79-876e-4f91-9821-b384e1169892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:53.192536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc59a79-876e-4f91-9821-b384e1169892 admin admin] PATH_INFO: `/v3/projects/39f7be9679a84619ad235ec1b0f83f93/users/9697987d044249fba621cb6e2b787c54/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:53.194128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89774e2c-83cb-47a0-a16b-b0135d839216 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:53.214195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0072c0-c55e-4853-be6a-1abbf291f0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:53.215418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0072c0-c55e-4853-be6a-1abbf291f0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:53.215724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0072c0-c55e-4853-be6a-1abbf291f0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:53.216097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0072c0-c55e-4853-be6a-1abbf291f0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:53.216857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0072c0-c55e-4853-be6a-1abbf291f0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:53.227459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0072c0-c55e-4853-be6a-1abbf291f0b0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:53.251124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc59a79-876e-4f91-9821-b384e1169892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39f7be9679a84619ad235ec1b0f83f93, user_id=9697987d044249fba621cb6e2b787c54, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:53.257953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc59a79-876e-4f91-9821-b384e1169892 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:53.312581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 867/1745] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:53 2022] PUT /identity/v3/projects/39f7be9679a84619ad235ec1b0f83f93/users/9697987d044249fba621cb6e2b787c54/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:53.316480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e6265-8b29-49fd-b5b9-2d265964791b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:53.317075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e6265-8b29-49fd-b5b9-2d265964791b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:53.317598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e6265-8b29-49fd-b5b9-2d265964791b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:53.319572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 879/1746] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:53 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:53.320908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588f9f-f82c-4d3f-84c5-7b6972a9fd99 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:53.321196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588f9f-f82c-4d3f-84c5-7b6972a9fd99 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:53.321515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66588f9f-f82c-4d3f-84c5-7b6972a9fd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:53.427137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d47e6265-8b29-49fd-b5b9-2d265964791b None None] MFA Rules not processed for user `cf865cb3c7fc432884e4e308c397c83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:53.431072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d47e6265-8b29-49fd-b5b9-2d265964791b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:53.462340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:53.494945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-66588f9f-f82c-4d3f-84c5-7b6972a9fd99 None None] MFA Rules not processed for user `9697987d044249fba621cb6e2b787c54`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:53.499561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-66588f9f-f82c-4d3f-84c5-7b6972a9fd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:53.797826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 868/1747] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:30:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:53.798903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e6265-8b29-49fd-b5b9-2d265964791b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:53.846180 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:53.947308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 880/1748] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 627 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:53.954853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66588f9f-f82c-4d3f-84c5-7b6972a9fd99 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:53.960683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-66588f9f-f82c-4d3f-84c5-7b6972a9fd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:54.000266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66588f9f-f82c-4d3f-84c5-7b6972a9fd99 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:54.022286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d318-c322-4ab9-8ddf-4e5d1112ea17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:54.023146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d318-c322-4ab9-8ddf-4e5d1112ea17 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.023444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d318-c322-4ab9-8ddf-4e5d1112ea17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.025077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d318-c322-4ab9-8ddf-4e5d1112ea17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.054067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e6265-8b29-49fd-b5b9-2d265964791b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:54.062915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0d318-c322-4ab9-8ddf-4e5d1112ea17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:54.066506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0d318-c322-4ab9-8ddf-4e5d1112ea17 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:54.072787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:54.074225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.074688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.075380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.080029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:54.082965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:54.094196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 881/1749] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:54.095279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d318-c322-4ab9-8ddf-4e5d1112ea17 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:54.143448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d318-c322-4ab9-8ddf-4e5d1112ea17 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:54.172623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:54.175065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.175331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 869/1750] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:53 2022] POST /identity/v3/users => generated 292 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:54.175805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.175805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.177139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:54.182512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:54.184962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:54.187993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:54.245807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ce20-d8a0-46fa-9737-2cd7290626cb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:54.262659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:54.263823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.265998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.266535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] PATH_INFO: `/v3/users/427ca0235a034cf28857336b915b2c58` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.272380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=427ca0235a034cf28857336b915b2c58)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:54.296430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 882/1751] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:30:54 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:54.299636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:54.304979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:54.305679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:54.400298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150feae9-b46e-45d4-b887-05daafb7492e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:54.407299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:54.408459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.409391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.410062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] PATH_INFO: `/v3/projects/2c610147005e48a3801c5f5f23067f11` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.411759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c610147005e48a3801c5f5f23067f11)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:54.414742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:54.485909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '427ca0235a034cf28857336b915b2c58'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.486637 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '427ca0235a034cf28857336b915b2c58'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.487271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '427ca0235a034cf28857336b915b2c58'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.505068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '427ca0235a034cf28857336b915b2c58'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.524318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c610147005e48a3801c5f5f23067f11', 'request_id': 'req-f7e94c87-f609-4847-9384-f8829a4a6dc1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.547726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c610147005e48a3801c5f5f23067f11', 'request_id': 'req-f7e94c87-f609-4847-9384-f8829a4a6dc1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.547951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '427ca0235a034cf28857336b915b2c58'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.566430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c610147005e48a3801c5f5f23067f11', 'request_id': 'req-f7e94c87-f609-4847-9384-f8829a4a6dc1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.596976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 870/1752] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:54 2022] DELETE /identity/v3/users/427ca0235a034cf28857336b915b2c58 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:54.598149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] The token cache is being invalidate because project 2c610147005e48a3801c5f5f23067f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:30:54.598650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.600173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:54.600324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:54.604061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:54.610316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 883/1753] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:54 2022] DELETE /identity/v3/projects/2c610147005e48a3801c5f5f23067f11 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:54.611922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:54.613803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:54.662266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:54.713444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:54.775627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e94c87-f609-4847-9384-f8829a4a6dc1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:54.782721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e972439-d929-4409-a62c-75314971f366 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:54.787504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:54.788935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.789223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.789528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.790195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:54.792245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:54.800836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb56758-ae51-4a84-b551-80c14f8b514d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:54.801936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb56758-ae51-4a84-b551-80c14f8b514d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.802300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb56758-ae51-4a84-b551-80c14f8b514d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.802659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb56758-ae51-4a84-b551-80c14f8b514d admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.803426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb56758-ae51-4a84-b551-80c14f8b514d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:54.808995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb56758-ae51-4a84-b551-80c14f8b514d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:54.833832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 871/1754] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:54.833832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312ca3-60da-4ea8-a5e3-95fe1931568f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.834198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312ca3-60da-4ea8-a5e3-95fe1931568f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.834511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac312ca3-60da-4ea8-a5e3-95fe1931568f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.838440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 884/1755] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:30:54 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:30:54.842332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:54.848593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:54.873280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ac312ca3-60da-4ea8-a5e3-95fe1931568f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:54.878613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ac312ca3-60da-4ea8-a5e3-95fe1931568f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:54.879747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5b5b5-06be-48ec-be8a-08513eb95a08 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:54.895615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:54.896377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:54.896672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:54.896936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] PATH_INFO: `/v3/users/a13a396749404009a1fe181e8290177b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:54.898326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13a396749404009a1fe181e8290177b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:54.900351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:54.902128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 872/1756] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:54.906976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac312ca3-60da-4ea8-a5e3-95fe1931568f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:54.920181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ac312ca3-60da-4ea8-a5e3-95fe1931568f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:55.010193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac312ca3-60da-4ea8-a5e3-95fe1931568f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:55.016568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ac78d-eece-4941-8503-c7babebd0cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:55.017445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ac78d-eece-4941-8503-c7babebd0cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:55.017871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ac78d-eece-4941-8503-c7babebd0cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:55.018219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ac78d-eece-4941-8503-c7babebd0cd6 admin admin] PATH_INFO: `/v3/projects/0febb7331e1c40b7b43d208136dfcc47/users/003741b3db6f4c5e852e3834e707a603/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:55.037236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ac78d-eece-4941-8503-c7babebd0cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0febb7331e1c40b7b43d208136dfcc47, user_id=003741b3db6f4c5e852e3834e707a603, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:55.040257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ac78d-eece-4941-8503-c7babebd0cd6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:55.050960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13a396749404009a1fe181e8290177b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:55.068018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 873/1757] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:54 2022] PUT /identity/v3/projects/0febb7331e1c40b7b43d208136dfcc47/users/003741b3db6f4c5e852e3834e707a603/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:55.068018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ac78d-eece-4941-8503-c7babebd0cd6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:55.097157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13a396749404009a1fe181e8290177b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:55.097947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13a396749404009a1fe181e8290177b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:55.120633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13a396749404009a1fe181e8290177b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:55.144425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13a396749404009a1fe181e8290177b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:55.188107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 885/1758] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:54 2022] DELETE /identity/v3/users/a13a396749404009a1fe181e8290177b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:55.188613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:55.194513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:55.393140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef1f13-8d3f-4c48-b296-6964464aa6cf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:55.397892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:55.398203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:55.398470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:55.398722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:55.410185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:55.477713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ac78d-eece-4941-8503-c7babebd0cd6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:55.615581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:55.654866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:55.657177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:55.708807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 886/1759] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:55.709767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:55.719058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:55.780826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c6e80-6c5f-4414-ad4f-b400399d6499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:55.781116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093864a-ac74-4618-ae88-609700d84b26 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:55.782477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c6e80-6c5f-4414-ad4f-b400399d6499 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:55.784594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c6e80-6c5f-4414-ad4f-b400399d6499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:55.788151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c6e80-6c5f-4414-ad4f-b400399d6499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:55.789675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c6e80-6c5f-4414-ad4f-b400399d6499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:55.802197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:55.802842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:55.802968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c6e80-6c5f-4414-ad4f-b400399d6499 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:55.803155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:55.803401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:55.804093 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:55.806002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:55.845475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 874/1760] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 781 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:55.853507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a3343-e269-4b06-99b1-ae9e348dac3e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:55.857674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 887/1761] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:55.858719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:55.859585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a3343-e269-4b06-99b1-ae9e348dac3e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:55.868938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a3343-e269-4b06-99b1-ae9e348dac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:55.872713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:55.920610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da197c-96ae-4326-b230-7b597d30858a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:55.936863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9c7a3343-e269-4b06-99b1-ae9e348dac3e None None] MFA Rules not processed for user `98de1ccbabc64162a29cbce2cfe11600`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:55.938045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9c7a3343-e269-4b06-99b1-ae9e348dac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:55.950073 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:55.953041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:55.953041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:55.953041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] PATH_INFO: `/v3/users/a57413c042d444d592d8f21b35c6da28` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:55.961799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57413c042d444d592d8f21b35c6da28)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:55.968808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:56.067887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:56.193599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57413c042d444d592d8f21b35c6da28'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:56.254581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57413c042d444d592d8f21b35c6da28'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:56.270907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57413c042d444d592d8f21b35c6da28'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:56.273713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57413c042d444d592d8f21b35c6da28'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:56.299675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 875/1762] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:56.303344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a3343-e269-4b06-99b1-ae9e348dac3e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:56.313214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57413c042d444d592d8f21b35c6da28'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:56.321343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9c7a3343-e269-4b06-99b1-ae9e348dac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:56.338605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 888/1763] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:55 2022] DELETE /identity/v3/users/a57413c042d444d592d8f21b35c6da28 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:56.341993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:56.354451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:56.416159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a3343-e269-4b06-99b1-ae9e348dac3e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:56.551923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bdec1-9232-4eb1-891c-c98a232dc179 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:56.579516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7d52f-ecab-469f-b88c-28ae5db8f10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:56.580232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7d52f-ecab-469f-b88c-28ae5db8f10d admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:56.580515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7d52f-ecab-469f-b88c-28ae5db8f10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:56.580799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7d52f-ecab-469f-b88c-28ae5db8f10d admin admin] PATH_INFO: `/v3/projects/0febb7331e1c40b7b43d208136dfcc47/users/003741b3db6f4c5e852e3834e707a603/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:56.599021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7d52f-ecab-469f-b88c-28ae5db8f10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0febb7331e1c40b7b43d208136dfcc47, user_id=003741b3db6f4c5e852e3834e707a603, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:56.615209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7d52f-ecab-469f-b88c-28ae5db8f10d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:56.678395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ca3ef-dd10-4fed-917c-25acd03c5845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:56.679105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ca3ef-dd10-4fed-917c-25acd03c5845 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:56.679452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ca3ef-dd10-4fed-917c-25acd03c5845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:56.679663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ca3ef-dd10-4fed-917c-25acd03c5845 admin admin] PATH_INFO: `/v3/projects/d2f57026c00a449e98ece599d95c2336/users/1dfe0ef9b5f049f1b7861818325782b4/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:56.681232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 889/1764] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:56 2022] PUT /identity/v3/projects/0febb7331e1c40b7b43d208136dfcc47/users/003741b3db6f4c5e852e3834e707a603/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:56.682862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7c756-2c3e-4b32-950d-56f4e5272eca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:56.688447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7c756-2c3e-4b32-950d-56f4e5272eca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:56.688969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7c756-2c3e-4b32-950d-56f4e5272eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:56.727158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ca3ef-dd10-4fed-917c-25acd03c5845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f57026c00a449e98ece599d95c2336, user_id=1dfe0ef9b5f049f1b7861818325782b4, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:56.737204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ca3ef-dd10-4fed-917c-25acd03c5845 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:56.772000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-22a7c756-2c3e-4b32-950d-56f4e5272eca None None] MFA Rules not processed for user `ae71fd7ba4334d0d813cbc0c06e3686e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:56.773367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 876/1765] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:56 2022] PUT /identity/v3/projects/d2f57026c00a449e98ece599d95c2336/users/1dfe0ef9b5f049f1b7861818325782b4/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:56.775209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a41b9d-57f4-40ed-9458-5464e3f509c7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:56.776774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-22a7c756-2c3e-4b32-950d-56f4e5272eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:56.778658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a41b9d-57f4-40ed-9458-5464e3f509c7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:56.778955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a41b9d-57f4-40ed-9458-5464e3f509c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:56.853072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d2a41b9d-57f4-40ed-9458-5464e3f509c7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:56.858637 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d2a41b9d-57f4-40ed-9458-5464e3f509c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:56.929097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:57.021522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 890/1766] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:57.040732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e056b1-9355-45ee-975b-45001e945b80 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:57.041032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e056b1-9355-45ee-975b-45001e945b80 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:57.041314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e056b1-9355-45ee-975b-45001e945b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:57.118489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 877/1767] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:57.119033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a41b9d-57f4-40ed-9458-5464e3f509c7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:57.146920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a41b9d-57f4-40ed-9458-5464e3f509c7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:57.152877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-96e056b1-9355-45ee-975b-45001e945b80 None None] MFA Rules not processed for user `003741b3db6f4c5e852e3834e707a603`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:57.154377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-96e056b1-9355-45ee-975b-45001e945b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:57.160094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a236a-e06f-4e7e-9576-c506676761a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:57.160775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a236a-e06f-4e7e-9576-c506676761a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:57.161068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a236a-e06f-4e7e-9576-c506676761a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:57.161350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a236a-e06f-4e7e-9576-c506676761a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:57.165523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a236a-e06f-4e7e-9576-c506676761a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:57.167228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a236a-e06f-4e7e-9576-c506676761a5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:57.200368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 878/1768] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:30:57.202510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a236a-e06f-4e7e-9576-c506676761a5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:57.229565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a236a-e06f-4e7e-9576-c506676761a5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:57.252374 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:57.253352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:57.253669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:57.253946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] PATH_INFO: `/v3/projects/d2f57026c00a449e98ece599d95c2336/users/1dfe0ef9b5f049f1b7861818325782b4/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:57.266750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f57026c00a449e98ece599d95c2336, user_id=1dfe0ef9b5f049f1b7861818325782b4, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:57.274267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:57.331763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 879/1769] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:30:57 2022] PUT /identity/v3/projects/d2f57026c00a449e98ece599d95c2336/users/1dfe0ef9b5f049f1b7861818325782b4/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:57.332777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:57.353397 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:57.402852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 891/1770] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:57.408700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e44cb-c97d-4692-99c9-68d7310f4c2f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:57.411158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e44cb-c97d-4692-99c9-68d7310f4c2f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:57.412584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e44cb-c97d-4692-99c9-68d7310f4c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:57.510044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:57.565358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910607e-ab8c-4f5d-bb54-be269b765af3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:57.573581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ee1e44cb-c97d-4692-99c9-68d7310f4c2f None None] MFA Rules not processed for user `1dfe0ef9b5f049f1b7861818325782b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:57.580548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:57.581220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:57.581613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:57.581895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:57.585032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ee1e44cb-c97d-4692-99c9-68d7310f4c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:57.846210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:57.847733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:57.870966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 880/1771] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:57.871880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:57.876327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:57.942831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 892/1772] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:30:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 539 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:57.945114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e44cb-c97d-4692-99c9-68d7310f4c2f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:57.950864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ee1e44cb-c97d-4692-99c9-68d7310f4c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:57.988040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e44cb-c97d-4692-99c9-68d7310f4c2f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:58.005254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:58.006645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:58.006959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:58.007228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] PATH_INFO: `/v3/projects/d873eee402274ab1996dbede8a022104` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:58.009177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d873eee402274ab1996dbede8a022104)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:58.011267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:58.154316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd873eee402274ab1996dbede8a022104', 'request_id': 'req-77a94984-b69f-4f9b-bbea-4aa6a5059b02'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:58.174041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:58.174041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd873eee402274ab1996dbede8a022104', 'request_id': 'req-77a94984-b69f-4f9b-bbea-4aa6a5059b02'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:58.191187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd873eee402274ab1996dbede8a022104', 'request_id': 'req-77a94984-b69f-4f9b-bbea-4aa6a5059b02'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:58.211796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] The token cache is being invalidate because project d873eee402274ab1996dbede8a022104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:30:58.212525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:58.214206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:58.220187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 893/1773] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:57 2022] DELETE /identity/v3/projects/d873eee402274ab1996dbede8a022104 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:58.220187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:58.221555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:58.261177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398ec55-4064-462f-aa50-f3c031a58695 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:58.315898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a94984-b69f-4f9b-bbea-4aa6a5059b02 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:58.333752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:58.334529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:58.334928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:58.335216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:58.341827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:58.493786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:58.499486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1ff64-ab71-426e-8e20-8369e368ce89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:58.499794 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1ff64-ab71-426e-8e20-8369e368ce89 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:58.500119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1ff64-ab71-426e-8e20-8369e368ce89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:58.500369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1ff64-ab71-426e-8e20-8369e368ce89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:58.504276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:58.517297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ccb1ff64-ab71-426e-8e20-8369e368ce89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:58.527005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 894/1774] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:58.531880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:58.537479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:58.673551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:58.729980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3b7a5-cf85-4ec2-a127-7c54899aada6 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:58.740500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1ff64-ab71-426e-8e20-8369e368ce89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:58.744579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1ff64-ab71-426e-8e20-8369e368ce89 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:58.765841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:58.766541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:58.766821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:58.767131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] PATH_INFO: `/v3/users/e4a07d3ace804951840c33c6bbbd9ec0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:58.779108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4a07d3ace804951840c33c6bbbd9ec0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:58.789250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:58.806124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 881/1775] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 934 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:30:58.807118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1591b-eafd-4784-bc31-a5e1bdf03856 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:58.807609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1591b-eafd-4784-bc31-a5e1bdf03856 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:58.807893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1591b-eafd-4784-bc31-a5e1bdf03856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:58.943254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4cf1591b-eafd-4784-bc31-a5e1bdf03856 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:30:58.944854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4cf1591b-eafd-4784-bc31-a5e1bdf03856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:58.991827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:30:59.031396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4a07d3ace804951840c33c6bbbd9ec0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:59.071215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4a07d3ace804951840c33c6bbbd9ec0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:59.074383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4a07d3ace804951840c33c6bbbd9ec0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:59.093099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4a07d3ace804951840c33c6bbbd9ec0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:59.145171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4a07d3ace804951840c33c6bbbd9ec0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:30:59.166115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 895/1776] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:30:58 2022] DELETE /identity/v3/users/e4a07d3ace804951840c33c6bbbd9ec0 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:30:59.167392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:59.172591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:59.226372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 882/1777] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:30:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:30:59.230440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1591b-eafd-4784-bc31-a5e1bdf03856 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:30:59.238429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4cf1591b-eafd-4784-bc31-a5e1bdf03856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:59.276259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1591b-eafd-4784-bc31-a5e1bdf03856 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:59.410914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d391a-8f06-4cca-910c-1964191a5634 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:30:59.443264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:59.444835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:59.446572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:59.453082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:59.467227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:30:59.627853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:30:59.631565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:30:59.641541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:30:59.643360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] PATH_INFO: `/v3/projects/f34743461ee24f538a9dd84157ecaac5` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:30:59.653370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f34743461ee24f538a9dd84157ecaac5)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:30:59.669437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:30:59.989509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:00.000422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:00.095827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 896/1778] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:30:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 927 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:00.102245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:00.107132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f34743461ee24f538a9dd84157ecaac5', 'request_id': 'req-b418e05e-b301-43f8-b496-4b82a8b8650e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:00.111298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:00.175893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f34743461ee24f538a9dd84157ecaac5', 'request_id': 'req-b418e05e-b301-43f8-b496-4b82a8b8650e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:00.227455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f34743461ee24f538a9dd84157ecaac5', 'request_id': 'req-b418e05e-b301-43f8-b496-4b82a8b8650e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:00.246788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:00.312578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] The token cache is being invalidate because project f34743461ee24f538a9dd84157ecaac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:00.314173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:00.320014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:00.344290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 883/1779] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:30:59 2022] DELETE /identity/v3/projects/f34743461ee24f538a9dd84157ecaac5 => generated 0 bytes in 1117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:00.351701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:00.358130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:00.720095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:00.804818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418e05e-b301-43f8-b496-4b82a8b8650e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:00.815193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:00.816328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:00.816831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:00.817387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] PATH_INFO: `/v3/users/2f2b7f027b5f4d7692ba3ad4d9593c18` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:00.820060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f2b7f027b5f4d7692ba3ad4d9593c18)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:00.824775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:00.839354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7bec99-96f8-4844-8f9d-e6de53698708 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:00.990630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4094e8-bc43-4d1a-88e2-fb54f63cf201 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:00.991675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4094e8-bc43-4d1a-88e2-fb54f63cf201 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:00.991907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4094e8-bc43-4d1a-88e2-fb54f63cf201 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:00.992302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4094e8-bc43-4d1a-88e2-fb54f63cf201 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:01.008775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7c4094e8-bc43-4d1a-88e2-fb54f63cf201 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:01.056526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f2b7f027b5f4d7692ba3ad4d9593c18'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:01.058820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f2b7f027b5f4d7692ba3ad4d9593c18'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:01.074693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f2b7f027b5f4d7692ba3ad4d9593c18'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:01.103611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f2b7f027b5f4d7692ba3ad4d9593c18'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:01.145183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e29b6b02-d8f0-4aa1-b464-de312c551af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f2b7f027b5f4d7692ba3ad4d9593c18'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:01.209140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 884/1780] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:00 2022] DELETE /identity/v3/users/2f2b7f027b5f4d7692ba3ad4d9593c18 => generated 0 bytes in 864 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:01.214418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b59e74-7d2d-4cfe-ba48-784c366423ad None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:01.214418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b59e74-7d2d-4cfe-ba48-784c366423ad None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:01.216716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b59e74-7d2d-4cfe-ba48-784c366423ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:01.273009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4094e8-bc43-4d1a-88e2-fb54f63cf201 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:01.284903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4094e8-bc43-4d1a-88e2-fb54f63cf201 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:01.312757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b0b59e74-7d2d-4cfe-ba48-784c366423ad None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:01.315784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b0b59e74-7d2d-4cfe-ba48-784c366423ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:01.333213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 897/1781] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:31:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:31:01.354524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb7096-5f1c-401d-bf8b-27e9952d7988 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:01.363892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb7096-5f1c-401d-bf8b-27e9952d7988 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:01.367395 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb7096-5f1c-401d-bf8b-27e9952d7988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:01.514805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-03eb7096-5f1c-401d-bf8b-27e9952d7988 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:01.517469 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-03eb7096-5f1c-401d-bf8b-27e9952d7988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:01.757645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 885/1782] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:01.760490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e5aa-e06e-4f18-91da-17d3d121fcd8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:01.761365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e5aa-e06e-4f18-91da-17d3d121fcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:01.761696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e5aa-e06e-4f18-91da-17d3d121fcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:01.950472 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e099e5aa-e06e-4f18-91da-17d3d121fcd8 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:01.962922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e099e5aa-e06e-4f18-91da-17d3d121fcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:02.019888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 886/1783] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:02.021137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099e5aa-e06e-4f18-91da-17d3d121fcd8 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:02.029359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e099e5aa-e06e-4f18-91da-17d3d121fcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:02.048122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 898/1784] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 714 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:02.049266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb7096-5f1c-401d-bf8b-27e9952d7988 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:02.053483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099e5aa-e06e-4f18-91da-17d3d121fcd8 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:02.066930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:02.068059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:02.068469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:02.069365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] PATH_INFO: `/v3/projects/33d7dd0b35db44d98d484f13e733a143` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:02.083908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb7096-5f1c-401d-bf8b-27e9952d7988 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:02.094167 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d7dd0b35db44d98d484f13e733a143)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:02.097777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:02.102254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:02.103325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:02.103806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:02.104273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:02.109543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:02.110316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:02.112109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:02.114455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:02.159686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 899/1785] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:02.171839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:02.214648 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765ce39-586f-422a-a639-0f5075cda980 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:02.254851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:02.255905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:02.256413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:02.256882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:02.258070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:02.274412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:02.338940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d7dd0b35db44d98d484f13e733a143', 'request_id': 'req-c666a0ad-9372-4afe-9efa-66d33a564a92'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:02.379195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:02.386629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d7dd0b35db44d98d484f13e733a143', 'request_id': 'req-c666a0ad-9372-4afe-9efa-66d33a564a92'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:02.428009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d7dd0b35db44d98d484f13e733a143', 'request_id': 'req-c666a0ad-9372-4afe-9efa-66d33a564a92'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:02.504530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 900/1786] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:02 2022] POST /identity/v3/projects => generated 387 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:02.512267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] The token cache is being invalidate because project 33d7dd0b35db44d98d484f13e733a143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:02.516438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:02.519894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:02.529247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:02.531807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:02.543823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 887/1787] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:02 2022] DELETE /identity/v3/projects/33d7dd0b35db44d98d484f13e733a143 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:02.555269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:02.579016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:02.733444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890a540-a0db-4be3-89bb-f85b3fa6ae77 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:02.741761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:02.743580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:02.744261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:02.745239 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:02.749974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:02.914226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666a0ad-9372-4afe-9efa-66d33a564a92 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:03.136248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:03.143608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:03.224673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 901/1788] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:03.230305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:03.243320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:03.322397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180609c-d89b-401b-9249-c2b45e5f766a service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:03.325766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:03.331610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:03.331949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:03.335595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] PATH_INFO: `/v3/projects/199c3cb5f1d843228356cbf4983158bb` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:03.337124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] RBAC: Authorizing `identity:delete_project(project_id=199c3cb5f1d843228356cbf4983158bb)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:03.342569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:03.346023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:03.347461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:03.347935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:03.348381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:03.349288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:03.351987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:03.441557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 902/1789] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:03 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:03.441557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:03.445740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:03.521859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:03.531817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8d7d0-a99d-47f8-959a-05906a12fa68 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:03.554844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199c3cb5f1d843228356cbf4983158bb', 'request_id': 'req-411a9217-0ecc-4711-a5aa-6bc94baab57f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:03.592791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199c3cb5f1d843228356cbf4983158bb', 'request_id': 'req-411a9217-0ecc-4711-a5aa-6bc94baab57f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:03.650533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199c3cb5f1d843228356cbf4983158bb', 'request_id': 'req-411a9217-0ecc-4711-a5aa-6bc94baab57f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:03.715513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] The token cache is being invalidate because project 199c3cb5f1d843228356cbf4983158bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:03.715937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:03.721208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:03.737905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 888/1790] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:02 2022] DELETE /identity/v3/projects/199c3cb5f1d843228356cbf4983158bb => generated 0 bytes in 1178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:03.737905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:03.740949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:04.038647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a9217-0ecc-4711-a5aa-6bc94baab57f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:04.063412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a2a46-68f1-4a50-aa3d-fa5c1d9080ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:04.065495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2a46-68f1-4a50-aa3d-fa5c1d9080ac admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:04.065495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2a46-68f1-4a50-aa3d-fa5c1d9080ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:04.066151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2a46-68f1-4a50-aa3d-fa5c1d9080ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:04.067509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a2a46-68f1-4a50-aa3d-fa5c1d9080ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:04.070769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a2a46-68f1-4a50-aa3d-fa5c1d9080ac admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:04.071421 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:04.072570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:04.075689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:04.076312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] PATH_INFO: `/v3/users/51e75998702847e2a39aca23c1ca53bd` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:04.086264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e75998702847e2a39aca23c1ca53bd)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:04.101540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:04.157546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 889/1791] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:04.160329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56bf24-53e8-4f1c-b1d4-bd448aa4f330 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:04.160891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56bf24-53e8-4f1c-b1d4-bd448aa4f330 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:04.166060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56bf24-53e8-4f1c-b1d4-bd448aa4f330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:04.246031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:04.275285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6a56bf24-53e8-4f1c-b1d4-bd448aa4f330 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:04.277848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6a56bf24-53e8-4f1c-b1d4-bd448aa4f330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:04.300971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e75998702847e2a39aca23c1ca53bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:04.325484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 890/1792] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:04.327181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56bf24-53e8-4f1c-b1d4-bd448aa4f330 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:04.364565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56bf24-53e8-4f1c-b1d4-bd448aa4f330 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:04.393081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4537476-a34f-469e-bcfb-978509b6f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:04.394183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537476-a34f-469e-bcfb-978509b6f30e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:04.394640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537476-a34f-469e-bcfb-978509b6f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:04.395113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537476-a34f-469e-bcfb-978509b6f30e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:04.396036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4537476-a34f-469e-bcfb-978509b6f30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:04.398009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4537476-a34f-469e-bcfb-978509b6f30e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:04.428482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e75998702847e2a39aca23c1ca53bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:04.429383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 891/1793] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:04.430397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e75998702847e2a39aca23c1ca53bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:04.431335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200e7bb-eb31-4d8c-9eac-4b54de85b10d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:04.441730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200e7bb-eb31-4d8c-9eac-4b54de85b10d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:04.442537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200e7bb-eb31-4d8c-9eac-4b54de85b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:04.484503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7200e7bb-eb31-4d8c-9eac-4b54de85b10d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:04.486177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7200e7bb-eb31-4d8c-9eac-4b54de85b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:04.508268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 892/1794] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:04.509313 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200e7bb-eb31-4d8c-9eac-4b54de85b10d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:04.509510 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e75998702847e2a39aca23c1ca53bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:04.515345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7200e7bb-eb31-4d8c-9eac-4b54de85b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:04.548205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200e7bb-eb31-4d8c-9eac-4b54de85b10d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:04.555779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5144c75-3488-4e81-b317-f7daa8f37507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:04.556534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144c75-3488-4e81-b317-f7daa8f37507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:04.556818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144c75-3488-4e81-b317-f7daa8f37507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:04.557152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144c75-3488-4e81-b317-f7daa8f37507 admin admin] PATH_INFO: `/v3/projects/b9009bc8238f44f0ba64769c8bef11d6/users/67bf2bcd856d4a148051cc4829628b6a/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:04.600443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5144c75-3488-4e81-b317-f7daa8f37507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9009bc8238f44f0ba64769c8bef11d6, user_id=67bf2bcd856d4a148051cc4829628b6a, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:04.603523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5144c75-3488-4e81-b317-f7daa8f37507 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:04.616162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c64ed883-6fc0-473a-a37e-010eeaa43d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e75998702847e2a39aca23c1ca53bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:04.647051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 893/1795] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:04 2022] PUT /identity/v3/projects/b9009bc8238f44f0ba64769c8bef11d6/users/67bf2bcd856d4a148051cc4829628b6a/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:04.648207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5144c75-3488-4e81-b317-f7daa8f37507 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:04.651038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 903/1796] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:03 2022] DELETE /identity/v3/users/51e75998702847e2a39aca23c1ca53bd => generated 0 bytes in 1213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:04.658262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04373f9d-bebd-48e9-9192-982b788b0c90 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:04.659318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04373f9d-bebd-48e9-9192-982b788b0c90 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:04.660139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04373f9d-bebd-48e9-9192-982b788b0c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:04.803614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-04373f9d-bebd-48e9-9192-982b788b0c90 None None] MFA Rules not processed for user `67bf2bcd856d4a148051cc4829628b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:04.806290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-04373f9d-bebd-48e9-9192-982b788b0c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:04.821270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5144c75-3488-4e81-b317-f7daa8f37507 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:04.865920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:05.074798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 904/1797] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:05.079860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04373f9d-bebd-48e9-9192-982b788b0c90 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:05.117901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:05.128552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:05.140483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:05.144707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:05.160960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:05.164509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:05.189763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:05.218647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04373f9d-bebd-48e9-9192-982b788b0c90 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:05.242816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04f35b-7340-4a1f-b824-e23545513e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:05.243827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04f35b-7340-4a1f-b824-e23545513e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:05.245190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04f35b-7340-4a1f-b824-e23545513e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:05.246926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04f35b-7340-4a1f-b824-e23545513e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:05.249269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04f35b-7340-4a1f-b824-e23545513e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:05.260973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04f35b-7340-4a1f-b824-e23545513e4b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:05.308146 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 894/1798] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 660 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:05.315130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:05.401108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 905/1799] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:05 2022] POST /identity/v3/projects => generated 387 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:05.406779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04f35b-7340-4a1f-b824-e23545513e4b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:05.425954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928c097-340d-4f06-bf58-27034fa6b037 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:05.694914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04f35b-7340-4a1f-b824-e23545513e4b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:05.736118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661d88c-156a-4cbf-a9bd-a0aad9572939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:05.736961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661d88c-156a-4cbf-a9bd-a0aad9572939 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:05.737247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661d88c-156a-4cbf-a9bd-a0aad9572939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:05.737593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661d88c-156a-4cbf-a9bd-a0aad9572939 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:05.738470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ee0d1-ae25-4fa5-8b28-7d776e04a48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:05.738978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ee0d1-ae25-4fa5-8b28-7d776e04a48c admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:05.739271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ee0d1-ae25-4fa5-8b28-7d776e04a48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:05.739557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ee0d1-ae25-4fa5-8b28-7d776e04a48c admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:05.740254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ee0d1-ae25-4fa5-8b28-7d776e04a48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:05.743246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661d88c-156a-4cbf-a9bd-a0aad9572939 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:05.758877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ee0d1-ae25-4fa5-8b28-7d776e04a48c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:05.759163 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661d88c-156a-4cbf-a9bd-a0aad9572939 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:05.821416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 906/1800] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:05 2022] POST /identity/v3/users => generated 300 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:05.821416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9143ef-c5d5-401f-87c3-804862880bec None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:05.824879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9143ef-c5d5-401f-87c3-804862880bec None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:05.825395 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9143ef-c5d5-401f-87c3-804862880bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:05.880634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 895/1801] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:05 2022] POST /identity/v3/projects => generated 385 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:05.880634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ee0d1-ae25-4fa5-8b28-7d776e04a48c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:05.889496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:05.975706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1f9143ef-c5d5-401f-87c3-804862880bec None None] MFA Rules not processed for user `cf865cb3c7fc432884e4e308c397c83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:05.983484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1f9143ef-c5d5-401f-87c3-804862880bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:06.132394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 907/1802] 10.222.0.65 () {58 vars in 1015 bytes} [Mon Oct 3 13:31:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:06.133491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9143ef-c5d5-401f-87c3-804862880bec None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:06.158188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9143ef-c5d5-401f-87c3-804862880bec None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:06.172579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ee0d1-ae25-4fa5-8b28-7d776e04a48c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:06.191589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0c71-89b5-4fa5-8548-b91e615709e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:06.192329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0c71-89b5-4fa5-8548-b91e615709e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:06.192731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0c71-89b5-4fa5-8548-b91e615709e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:06.193030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0c71-89b5-4fa5-8548-b91e615709e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:06.195096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df0c71-89b5-4fa5-8548-b91e615709e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:06.196134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df0c71-89b5-4fa5-8548-b91e615709e3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:06.236589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409b258-8e32-4ac2-b38c-11df8f16861f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:06.257235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b258-8e32-4ac2-b38c-11df8f16861f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:06.257235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b258-8e32-4ac2-b38c-11df8f16861f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:06.257235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409b258-8e32-4ac2-b38c-11df8f16861f admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:06.257235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409b258-8e32-4ac2-b38c-11df8f16861f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:06.264935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409b258-8e32-4ac2-b38c-11df8f16861f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:06.278128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 908/1803] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:06 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:06.279241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0c71-89b5-4fa5-8548-b91e615709e3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:06.336473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0c71-89b5-4fa5-8548-b91e615709e3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:06.353486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 896/1804] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:06.355198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409b258-8e32-4ac2-b38c-11df8f16861f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:06.365333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7dd7-ee39-430f-bdc6-103c54b423aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:06.366406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7dd7-ee39-430f-bdc6-103c54b423aa admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:06.366704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7dd7-ee39-430f-bdc6-103c54b423aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:06.366990 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7dd7-ee39-430f-bdc6-103c54b423aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:06.368436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7dd7-ee39-430f-bdc6-103c54b423aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:06.370689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7dd7-ee39-430f-bdc6-103c54b423aa admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:06.390874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 909/1805] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:06.395968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409b258-8e32-4ac2-b38c-11df8f16861f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:06.424594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7dd7-ee39-430f-bdc6-103c54b423aa admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:06.432694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1835-3c6e-43e2-a812-aa30a862f6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:06.433867 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1835-3c6e-43e2-a812-aa30a862f6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:06.434338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1835-3c6e-43e2-a812-aa30a862f6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:06.434827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1835-3c6e-43e2-a812-aa30a862f6a3 admin admin] PATH_INFO: `/v3/projects/66b13b68edfb48609eee2d1bb2867f37/users/d7b8633fa8e042ca9122257aac50251e/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:06.456824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7dd7-ee39-430f-bdc6-103c54b423aa admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:06.469379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce9acc-8df0-4ccb-9730-3d64374f2ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:06.470661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce9acc-8df0-4ccb-9730-3d64374f2ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:06.471174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce9acc-8df0-4ccb-9730-3d64374f2ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:06.471749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce9acc-8df0-4ccb-9730-3d64374f2ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:06.473063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce9acc-8df0-4ccb-9730-3d64374f2ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:06.476027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce9acc-8df0-4ccb-9730-3d64374f2ffe admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:06.482018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de1835-3c6e-43e2-a812-aa30a862f6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b13b68edfb48609eee2d1bb2867f37, user_id=d7b8633fa8e042ca9122257aac50251e, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:06.487497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de1835-3c6e-43e2-a812-aa30a862f6a3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:06.495924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 910/1806] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:06.504199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce9acc-8df0-4ccb-9730-3d64374f2ffe admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:06.523889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce9acc-8df0-4ccb-9730-3d64374f2ffe admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:06.532572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 897/1807] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:06 2022] PUT /identity/v3/projects/66b13b68edfb48609eee2d1bb2867f37/users/d7b8633fa8e042ca9122257aac50251e/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:06.540231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1835-3c6e-43e2-a812-aa30a862f6a3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:06.628592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:06.791932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1835-3c6e-43e2-a812-aa30a862f6a3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:06.796533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f181c1-4abe-46b6-a48c-6be49ec1e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:06.798697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f181c1-4abe-46b6-a48c-6be49ec1e3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:06.799219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f181c1-4abe-46b6-a48c-6be49ec1e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:06.799712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f181c1-4abe-46b6-a48c-6be49ec1e3fc admin admin] PATH_INFO: `/v3/projects/9b69262fc60a4b6a926e0776a50c71ac/users/fdbd4bb7bad741afa0b2242b6bed4f1c/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:06.801729 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:06.802680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:06.803152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:06.803725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:06.804902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:06.807030 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:06.821918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 898/1808] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:31:06.822982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:06.825281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:06.848553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeebac-a40b-4f37-9390-19efb6c300d7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:06.849278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f181c1-4abe-46b6-a48c-6be49ec1e3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b69262fc60a4b6a926e0776a50c71ac, user_id=fdbd4bb7bad741afa0b2242b6bed4f1c, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:06.855228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:06.856011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:06.856376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:06.856673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f181c1-4abe-46b6-a48c-6be49ec1e3fc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:06.856845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] PATH_INFO: `/v3/projects/c9a23b3336e84fc5a490e2a07fed6c13` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:06.858588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a23b3336e84fc5a490e2a07fed6c13)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:06.860460 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:06.886959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 911/1809] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:06 2022] PUT /identity/v3/projects/9b69262fc60a4b6a926e0776a50c71ac/users/fdbd4bb7bad741afa0b2242b6bed4f1c/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:06.902164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f181c1-4abe-46b6-a48c-6be49ec1e3fc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:06.962215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a23b3336e84fc5a490e2a07fed6c13', 'request_id': 'req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:06.977271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a23b3336e84fc5a490e2a07fed6c13', 'request_id': 'req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:06.989393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a23b3336e84fc5a490e2a07fed6c13', 'request_id': 'req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.022001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] The token cache is being invalidate because project c9a23b3336e84fc5a490e2a07fed6c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:07.022001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.026483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7bbea6ab-2c9b-4500-9bf9-9bb1849867fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.028728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 899/1810] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:06 2022] DELETE /identity/v3/projects/c9a23b3336e84fc5a490e2a07fed6c13 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:07.030598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531d43-af81-4339-a495-57baff216f2b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:07.030883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531d43-af81-4339-a495-57baff216f2b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:07.031153 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad531d43-af81-4339-a495-57baff216f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:07.043906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:07.049169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f181c1-4abe-46b6-a48c-6be49ec1e3fc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:07.073356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ad531d43-af81-4339-a495-57baff216f2b None None] MFA Rules not processed for user `fdbd4bb7bad741afa0b2242b6bed4f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:07.074863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad531d43-af81-4339-a495-57baff216f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:07.137925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:07.138723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:07.139367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:07.139725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] PATH_INFO: `/v3/projects/66b13b68edfb48609eee2d1bb2867f37/users/d7b8633fa8e042ca9122257aac50251e/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:07.142179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b13b68edfb48609eee2d1bb2867f37, user_id=d7b8633fa8e042ca9122257aac50251e, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:07.146182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:07.166632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 912/1811] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:06 2022] PUT /identity/v3/projects/66b13b68edfb48609eee2d1bb2867f37/users/d7b8633fa8e042ca9122257aac50251e/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:07.168798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:07.180423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:07.250225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 900/1812] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:07.251938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad531d43-af81-4339-a495-57baff216f2b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:07.254631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad531d43-af81-4339-a495-57baff216f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:07.276393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad531d43-af81-4339-a495-57baff216f2b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:07.299334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03eec2-520a-4d97-9fb2-9ccb02309266 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:07.305128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:07.312947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:07.313967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:07.314262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:07.314550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] PATH_INFO: `/v3/users/003741b3db6f4c5e852e3834e707a603` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:07.316308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] RBAC: Authorizing `identity:delete_user(user_id=003741b3db6f4c5e852e3834e707a603)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:07.318991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:07.381253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:07.382173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:07.382527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:07.382925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] PATH_INFO: `/v3/projects/74e7a503942c4b3d8ab7955f98b2d794` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:07.384507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e7a503942c4b3d8ab7955f98b2d794)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:07.386987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:07.432476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003741b3db6f4c5e852e3834e707a603'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.463228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003741b3db6f4c5e852e3834e707a603'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.464028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003741b3db6f4c5e852e3834e707a603'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.476242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003741b3db6f4c5e852e3834e707a603'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.480193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e7a503942c4b3d8ab7955f98b2d794', 'request_id': 'req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.489028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2e1b368-7d21-426a-a989-1444268b913d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003741b3db6f4c5e852e3834e707a603'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.495440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e7a503942c4b3d8ab7955f98b2d794', 'request_id': 'req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.510497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 913/1813] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:07 2022] DELETE /identity/v3/users/003741b3db6f4c5e852e3834e707a603 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:07.510497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7464d-e22f-4081-bdcd-c617fb6d7d55 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:07.510497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7464d-e22f-4081-bdcd-c617fb6d7d55 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:07.510497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7464d-e22f-4081-bdcd-c617fb6d7d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:07.512819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e7a503942c4b3d8ab7955f98b2d794', 'request_id': 'req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.539641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] The token cache is being invalidate because project 74e7a503942c4b3d8ab7955f98b2d794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:07.540373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.542483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:07.545736 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 901/1814] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:07 2022] DELETE /identity/v3/projects/74e7a503942c4b3d8ab7955f98b2d794 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:07.547190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:07.548705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:07.589620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b8e7464d-e22f-4081-bdcd-c617fb6d7d55 None None] MFA Rules not processed for user `d7b8633fa8e042ca9122257aac50251e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:07.594784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b8e7464d-e22f-4081-bdcd-c617fb6d7d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:07.611120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:07.658647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffc16b-90a5-4fe5-96fa-d56aca19dbbc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:07.781499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 914/1815] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:07.781499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9ea98-699b-4cb3-8ab6-980973a7264c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:07.781499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9ea98-699b-4cb3-8ab6-980973a7264c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:07.782164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9ea98-699b-4cb3-8ab6-980973a7264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:07.869249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d0b9ea98-699b-4cb3-8ab6-980973a7264c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:07.872591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d0b9ea98-699b-4cb3-8ab6-980973a7264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:07.878886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4b63f-0391-4aa4-bb4f-fbe946b3b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:07.879559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b63f-0391-4aa4-bb4f-fbe946b3b09c admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:07.879834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b63f-0391-4aa4-bb4f-fbe946b3b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:07.880116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b63f-0391-4aa4-bb4f-fbe946b3b09c admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:07.880926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4b63f-0391-4aa4-bb4f-fbe946b3b09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:07.882892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4b63f-0391-4aa4-bb4f-fbe946b3b09c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:07.892462 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 915/1816] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:07.898497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9ea98-699b-4cb3-8ab6-980973a7264c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:07.901109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d0b9ea98-699b-4cb3-8ab6-980973a7264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:07.903336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:08.004243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 902/1817] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:07 2022] POST /identity/v3/projects => generated 385 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:08.007090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4d3d3-79ab-4c13-9649-3e3b9d4eca6e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:08.007712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4d3d3-79ab-4c13-9649-3e3b9d4eca6e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:08.017989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4d3d3-79ab-4c13-9649-3e3b9d4eca6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:08.086529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-74d4d3d3-79ab-4c13-9649-3e3b9d4eca6e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:08.090928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9ea98-699b-4cb3-8ab6-980973a7264c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:08.094814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74d4d3d3-79ab-4c13-9649-3e3b9d4eca6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:08.259400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c988f4-5c3b-4c58-a566-86c32abaee69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:08.260062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c988f4-5c3b-4c58-a566-86c32abaee69 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:08.260457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c988f4-5c3b-4c58-a566-86c32abaee69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:08.260738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c988f4-5c3b-4c58-a566-86c32abaee69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:08.263350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 903/1818] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:08.266262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4d3d3-79ab-4c13-9649-3e3b9d4eca6e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:08.283062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4d3d3-79ab-4c13-9649-3e3b9d4eca6e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:08.291724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8036365-29e8-4c5a-b25e-3caf94b5c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:08.292644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8036365-29e8-4c5a-b25e-3caf94b5c385 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:08.293212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8036365-29e8-4c5a-b25e-3caf94b5c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:08.293699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8036365-29e8-4c5a-b25e-3caf94b5c385 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:08.294834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8036365-29e8-4c5a-b25e-3caf94b5c385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:08.299570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8036365-29e8-4c5a-b25e-3caf94b5c385 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:08.363307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 904/1819] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:08 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:08.364751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8036365-29e8-4c5a-b25e-3caf94b5c385 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:08.385298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8036365-29e8-4c5a-b25e-3caf94b5c385 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:08.394858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:08.395884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:08.396583 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:08.397088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:08.398351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:08.398919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:08.402350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:08.430953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 905/1820] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:08.432536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:08.436446 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:08.459920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c988f4-5c3b-4c58-a566-86c32abaee69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:08.462776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c988f4-5c3b-4c58-a566-86c32abaee69 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:08.463723 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48af8c-8ca5-4158-ac1a-5c81a35a169f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:08.476848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:08.478269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:08.478881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:08.480112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] PATH_INFO: `/v3/projects/0febb7331e1c40b7b43d208136dfcc47` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:08.483390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0febb7331e1c40b7b43d208136dfcc47)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:08.487356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 916/1821] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:08.487975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:08.499836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c988f4-5c3b-4c58-a566-86c32abaee69 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:08.509270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:08.528378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c988f4-5c3b-4c58-a566-86c32abaee69 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:08.557509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657deac7-2251-47ae-bcf6-af063c302ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:08.557509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657deac7-2251-47ae-bcf6-af063c302ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:08.557509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657deac7-2251-47ae-bcf6-af063c302ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:08.557509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657deac7-2251-47ae-bcf6-af063c302ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:08.558651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657deac7-2251-47ae-bcf6-af063c302ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:08.563333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657deac7-2251-47ae-bcf6-af063c302ae6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:08.595736 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0febb7331e1c40b7b43d208136dfcc47', 'request_id': 'req-18e3c4cd-947f-4f1e-9948-bcf978712d98'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:08.610065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0febb7331e1c40b7b43d208136dfcc47', 'request_id': 'req-18e3c4cd-947f-4f1e-9948-bcf978712d98'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:08.615529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 917/1822] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:08.617479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657deac7-2251-47ae-bcf6-af063c302ae6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:08.624001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0febb7331e1c40b7b43d208136dfcc47', 'request_id': 'req-18e3c4cd-947f-4f1e-9948-bcf978712d98'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:08.642739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657deac7-2251-47ae-bcf6-af063c302ae6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:08.686764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] The token cache is being invalidate because project 0febb7331e1c40b7b43d208136dfcc47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:08.687539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:08.689177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:08.692234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 906/1823] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:08 2022] DELETE /identity/v3/projects/0febb7331e1c40b7b43d208136dfcc47 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:08.694194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:08.696669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:08.818591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd7944-46c6-4a32-b293-b7b78307f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:08.819492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd7944-46c6-4a32-b293-b7b78307f86a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:08.820006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd7944-46c6-4a32-b293-b7b78307f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:08.820385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd7944-46c6-4a32-b293-b7b78307f86a admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:08.821682 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd7944-46c6-4a32-b293-b7b78307f86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:08.830654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd7944-46c6-4a32-b293-b7b78307f86a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:08.848366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:08.897418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 918/1824] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:08 2022] POST /identity/v3/projects => generated 375 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:08.897418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443f5a9-2f4d-424e-9365-0602c0d0d049 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:08.897418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443f5a9-2f4d-424e-9365-0602c0d0d049 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:08.897418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443f5a9-2f4d-424e-9365-0602c0d0d049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:08.898142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3c4cd-947f-4f1e-9948-bcf978712d98 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:08.928489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d443f5a9-2f4d-424e-9365-0602c0d0d049 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:08.929951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d443f5a9-2f4d-424e-9365-0602c0d0d049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:09.026623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17223a4-f2d6-41de-8b01-64d8723fe5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:09.027627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17223a4-f2d6-41de-8b01-64d8723fe5f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.027917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17223a4-f2d6-41de-8b01-64d8723fe5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.028192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17223a4-f2d6-41de-8b01-64d8723fe5f0 admin admin] PATH_INFO: `/v3/projects/f1cd2491d6c44a178ed7ac5336d6e269/users/e69662ac9b8c48dd8c265b0c95478203/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:09.074485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 919/1825] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:09.074485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443f5a9-2f4d-424e-9365-0602c0d0d049 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:09.074784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17223a4-f2d6-41de-8b01-64d8723fe5f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1cd2491d6c44a178ed7ac5336d6e269, user_id=e69662ac9b8c48dd8c265b0c95478203, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:09.074868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d443f5a9-2f4d-424e-9365-0602c0d0d049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:09.080975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17223a4-f2d6-41de-8b01-64d8723fe5f0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:09.125692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 907/1826] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:08 2022] PUT /identity/v3/projects/f1cd2491d6c44a178ed7ac5336d6e269/users/e69662ac9b8c48dd8c265b0c95478203/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:09.131221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeff592-e4dd-4541-b6f2-9eba6357fd94 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.131559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeff592-e4dd-4541-b6f2-9eba6357fd94 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.131803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeff592-e4dd-4541-b6f2-9eba6357fd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:09.134882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443f5a9-2f4d-424e-9365-0602c0d0d049 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:09.179923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4eeff592-e4dd-4541-b6f2-9eba6357fd94 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:09.182633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4eeff592-e4dd-4541-b6f2-9eba6357fd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:09.297745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c1573-7cb7-4869-b03a-a2581553256c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:09.298789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c1573-7cb7-4869-b03a-a2581553256c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.299735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c1573-7cb7-4869-b03a-a2581553256c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.300110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c1573-7cb7-4869-b03a-a2581553256c admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:09.301490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c1573-7cb7-4869-b03a-a2581553256c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:09.304669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c1573-7cb7-4869-b03a-a2581553256c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:09.363818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 920/1827] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:09 2022] POST /identity/v3/users => generated 288 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:09.368387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c1573-7cb7-4869-b03a-a2581553256c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:09.415842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c1573-7cb7-4869-b03a-a2581553256c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:09.430100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fb6bd-b6a3-4340-81fe-4e65858532bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:09.436494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fb6bd-b6a3-4340-81fe-4e65858532bc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.440298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fb6bd-b6a3-4340-81fe-4e65858532bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.440854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fb6bd-b6a3-4340-81fe-4e65858532bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:09.442804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fb6bd-b6a3-4340-81fe-4e65858532bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:09.452147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fb6bd-b6a3-4340-81fe-4e65858532bc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:09.471714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:09.487684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 908/1828] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:09.488759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeff592-e4dd-4541-b6f2-9eba6357fd94 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:09.506635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeff592-e4dd-4541-b6f2-9eba6357fd94 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:09.526358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:09.527474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.527972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.528541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:09.531329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:09.533592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:09.557084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 921/1829] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:09.564612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fb6bd-b6a3-4340-81fe-4e65858532bc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:09.586642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 909/1830] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:09.586642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:09.587202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:09.634585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fb6bd-b6a3-4340-81fe-4e65858532bc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:09.657495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:09.657495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.657495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.657495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:09.658301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:09.665696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:09.668511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:09.686659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 922/1831] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:09.688482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:09.692115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:09.831027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659fbaf-e114-447f-abb3-ed35cbb4deb1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:09.839183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:09.840535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.841333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.842090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:09.847399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:09.862656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:09.864494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:09.870801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:09.880069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8477a-c6fe-42b7-8764-141b93317c52 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:09.881036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 910/1832] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:09.890276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1e36d-663c-42e7-920b-58e3ade953b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:09.891386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1e36d-663c-42e7-920b-58e3ade953b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.892063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1e36d-663c-42e7-920b-58e3ade953b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.892894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1e36d-663c-42e7-920b-58e3ade953b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:09.895213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:09.904524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:09.914848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1e36d-663c-42e7-920b-58e3ade953b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:09.924629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1e36d-663c-42e7-920b-58e3ade953b1 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:09.960497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb69787-66c5-42a3-a753-3b9c8a1df3e3 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:09.964954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 923/1833] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:09.974097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1e36d-663c-42e7-920b-58e3ade953b1 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:09.992983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0331b-bd4a-4180-9336-33b2c7a27df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:09.995526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0331b-bd4a-4180-9336-33b2c7a27df1 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:09.997425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0331b-bd4a-4180-9336-33b2c7a27df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:09.998717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0331b-bd4a-4180-9336-33b2c7a27df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:10.029194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0331b-bd4a-4180-9336-33b2c7a27df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:10.040768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0331b-bd4a-4180-9336-33b2c7a27df1 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:10.055904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 911/1834] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:10.057027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0331b-bd4a-4180-9336-33b2c7a27df1 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:10.059886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1e36d-663c-42e7-920b-58e3ade953b1 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:10.085067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3b08-2507-41ac-b66d-c8d082210090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:10.086390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3b08-2507-41ac-b66d-c8d082210090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:10.086891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3b08-2507-41ac-b66d-c8d082210090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:10.087402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f3b08-2507-41ac-b66d-c8d082210090 admin admin] PATH_INFO: `/v3/projects/f1cd2491d6c44a178ed7ac5336d6e269/users/e69662ac9b8c48dd8c265b0c95478203/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:10.092939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3b08-2507-41ac-b66d-c8d082210090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1cd2491d6c44a178ed7ac5336d6e269, user_id=e69662ac9b8c48dd8c265b0c95478203, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:10.096845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0331b-bd4a-4180-9336-33b2c7a27df1 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:10.099213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f3b08-2507-41ac-b66d-c8d082210090 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:10.114170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc42b5-be00-4be3-a4cc-da303bc684db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:10.116983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc42b5-be00-4be3-a4cc-da303bc684db admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:10.120021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc42b5-be00-4be3-a4cc-da303bc684db admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:10.122375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc42b5-be00-4be3-a4cc-da303bc684db admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:10.127064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc42b5-be00-4be3-a4cc-da303bc684db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:10.132919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dc42b5-be00-4be3-a4cc-da303bc684db admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:10.138000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 924/1835] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:09 2022] PUT /identity/v3/projects/f1cd2491d6c44a178ed7ac5336d6e269/users/e69662ac9b8c48dd8c265b0c95478203/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:10.140182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3b08-2507-41ac-b66d-c8d082210090 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:10.209807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 912/1836] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:10.212861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089e20c-5138-4480-8177-01e6bba1d386 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:10.213351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089e20c-5138-4480-8177-01e6bba1d386 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:10.238556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089e20c-5138-4480-8177-01e6bba1d386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:10.487611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1089e20c-5138-4480-8177-01e6bba1d386 None None] MFA Rules not processed for user `e69662ac9b8c48dd8c265b0c95478203`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:10.494540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1089e20c-5138-4480-8177-01e6bba1d386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:10.528129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f3b08-2507-41ac-b66d-c8d082210090 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:10.540833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9c08b-d432-4e11-8386-f3400d84c820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:10.543026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9c08b-d432-4e11-8386-f3400d84c820 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:10.543534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9c08b-d432-4e11-8386-f3400d84c820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:10.544015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9c08b-d432-4e11-8386-f3400d84c820 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:10.545869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9c08b-d432-4e11-8386-f3400d84c820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:10.549705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9c08b-d432-4e11-8386-f3400d84c820 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:10.634946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 925/1837] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:10 2022] POST /identity/v3/projects => generated 357 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:10.636378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9c08b-d432-4e11-8386-f3400d84c820 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:10.924448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:10.942339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9c08b-d432-4e11-8386-f3400d84c820 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:10.984052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f537356-4d93-48da-b89c-be6c4206e84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:10.985235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537356-4d93-48da-b89c-be6c4206e84f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:10.991775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537356-4d93-48da-b89c-be6c4206e84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:10.992656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f537356-4d93-48da-b89c-be6c4206e84f admin admin] PATH_INFO: `/v3/projects/6986cb28eb3f47a5a001115df4c1e4b7/users/61661d1fe90d4d46b59ae09c178a4c9d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:11.045575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 913/1838] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 834 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:11.049815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089e20c-5138-4480-8177-01e6bba1d386 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:11.056877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1089e20c-5138-4480-8177-01e6bba1d386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:11.080136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f537356-4d93-48da-b89c-be6c4206e84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6986cb28eb3f47a5a001115df4c1e4b7, user_id=61661d1fe90d4d46b59ae09c178a4c9d, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:11.093822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f537356-4d93-48da-b89c-be6c4206e84f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:11.111343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089e20c-5138-4480-8177-01e6bba1d386 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:11.128688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:11.129368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:11.132501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:11.132863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] PATH_INFO: `/v3/users/67bf2bcd856d4a148051cc4829628b6a` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:11.137040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67bf2bcd856d4a148051cc4829628b6a)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:11.145284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:11.221760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 926/1839] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:10 2022] PUT /identity/v3/projects/6986cb28eb3f47a5a001115df4c1e4b7/users/61661d1fe90d4d46b59ae09c178a4c9d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:11.246542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f537356-4d93-48da-b89c-be6c4206e84f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:11.316460 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67bf2bcd856d4a148051cc4829628b6a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:11.319526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67bf2bcd856d4a148051cc4829628b6a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:11.322205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67bf2bcd856d4a148051cc4829628b6a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:11.354561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67bf2bcd856d4a148051cc4829628b6a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:11.447006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67bf2bcd856d4a148051cc4829628b6a'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:11.501854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f537356-4d93-48da-b89c-be6c4206e84f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:11.513207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ee8a6-0ca5-4ea3-9738-50d18df760c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:11.514510 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ee8a6-0ca5-4ea3-9738-50d18df760c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:11.515633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ee8a6-0ca5-4ea3-9738-50d18df760c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:11.516454 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ee8a6-0ca5-4ea3-9738-50d18df760c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:11.518549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ee8a6-0ca5-4ea3-9738-50d18df760c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:11.525082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ee8a6-0ca5-4ea3-9738-50d18df760c5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:11.526406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 914/1840] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:11 2022] DELETE /identity/v3/users/67bf2bcd856d4a148051cc4829628b6a => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:11.533940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:11.553215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:11.642220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 927/1841] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:11 2022] POST /identity/v3/users => generated 279 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:11.646313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bc729-be71-4aa0-9aef-cda10cf990c9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:11.650692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bc729-be71-4aa0-9aef-cda10cf990c9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:11.655154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bc729-be71-4aa0-9aef-cda10cf990c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:11.761818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:11.807607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3b5bc729-be71-4aa0-9aef-cda10cf990c9 None None] MFA Rules not processed for user `61661d1fe90d4d46b59ae09c178a4c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:11.815585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3b5bc729-be71-4aa0-9aef-cda10cf990c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:11.854815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921d082-a1f3-44a6-824d-8d48c8f1a923 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:11.869868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:11.872561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:11.875828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:11.876594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:11.884438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:12.111062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 928/1842] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:12.111062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fee3f0-6484-4f8c-bcb5-27d028a30c14 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:12.112674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fee3f0-6484-4f8c-bcb5-27d028a30c14 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:12.115694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fee3f0-6484-4f8c-bcb5-27d028a30c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:12.195877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f8fee3f0-6484-4f8c-bcb5-27d028a30c14 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:12.200591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f8fee3f0-6484-4f8c-bcb5-27d028a30c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:12.207554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:12.212042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:12.281121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 915/1843] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 748 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:12.284160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:12.290128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:12.403618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:12.439117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 929/1844] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:12.446017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fee3f0-6484-4f8c-bcb5-27d028a30c14 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:12.455030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f8fee3f0-6484-4f8c-bcb5-27d028a30c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:12.542505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fee3f0-6484-4f8c-bcb5-27d028a30c14 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:12.571308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:12.580923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:12.582178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:12.584757 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:12.590054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:12.605520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:12.687641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c96b4-a91c-46d6-b04f-37ac8d6af89b service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:12.719160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:12.724706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 930/1845] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:12 2022] POST /identity/v3/projects => generated 375 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:12.725292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:12.725681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:12.726142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:12.726999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:12.727287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:12.735980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:12.740406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:12.805839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 916/1846] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 523 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:12.824987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:12.840193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:12.922998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002c84b-74f2-4d4c-9136-640d5ce6d9b8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:12.998978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f23c1-1112-4be4-baa9-2c114c695a53 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:13.027701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac4b23-5778-4d80-b662-bf48a5169ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:13.029056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac4b23-5778-4d80-b662-bf48a5169ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:13.032088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac4b23-5778-4d80-b662-bf48a5169ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:13.033874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac4b23-5778-4d80-b662-bf48a5169ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:13.246137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:13.246974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:13.247357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:13.250053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] PATH_INFO: `/v3/users/fdbd4bb7bad741afa0b2242b6bed4f1c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:13.254739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdbd4bb7bad741afa0b2242b6bed4f1c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:13.257488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:13.352695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:13.414039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac4b23-5778-4d80-b662-bf48a5169ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:13.415556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac4b23-5778-4d80-b662-bf48a5169ca7 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:13.452835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 931/1847] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:13.455175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac4b23-5778-4d80-b662-bf48a5169ca7 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:13.483028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdbd4bb7bad741afa0b2242b6bed4f1c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:13.484594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdbd4bb7bad741afa0b2242b6bed4f1c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:13.486443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac4b23-5778-4d80-b662-bf48a5169ca7 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:13.487041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdbd4bb7bad741afa0b2242b6bed4f1c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:13.501744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:13.503974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:13.504277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:13.506843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:13.510840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:13.519013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:13.520192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdbd4bb7bad741afa0b2242b6bed4f1c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:13.582094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 932/1848] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:13 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:13.585890 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:13.590278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:13.607032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b224fb28-f46d-4c02-bfe1-a7a8de0d1693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdbd4bb7bad741afa0b2242b6bed4f1c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:13.629273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba2972-2280-40e9-a2b2-95160a7a03c1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:13.648641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a783b9-9109-4cb8-86ca-18c49202db15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:13.649223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a783b9-9109-4cb8-86ca-18c49202db15 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:13.650524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a783b9-9109-4cb8-86ca-18c49202db15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:13.651225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a783b9-9109-4cb8-86ca-18c49202db15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:13.652451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a783b9-9109-4cb8-86ca-18c49202db15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:13.654217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a783b9-9109-4cb8-86ca-18c49202db15 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:13.666455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 917/1849] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:12 2022] DELETE /identity/v3/users/fdbd4bb7bad741afa0b2242b6bed4f1c => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:13.666455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627cda3-c9b0-42ca-95de-0841ecc693bb None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:13.666455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627cda3-c9b0-42ca-95de-0841ecc693bb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:13.666455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627cda3-c9b0-42ca-95de-0841ecc693bb None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:13.674666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 918/1850] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:31:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:31:13.677287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627cda3-c9b0-42ca-95de-0841ecc693bb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:13.688810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1627cda3-c9b0-42ca-95de-0841ecc693bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:13.703370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 933/1851] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:13.703592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4185d-f1db-4c9d-b4f2-08dab336a852 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:13.703864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4185d-f1db-4c9d-b4f2-08dab336a852 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:13.707127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4185d-f1db-4c9d-b4f2-08dab336a852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:13.789832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7bf4185d-f1db-4c9d-b4f2-08dab336a852 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:13.792518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7bf4185d-f1db-4c9d-b4f2-08dab336a852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:13.948715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627cda3-c9b0-42ca-95de-0841ecc693bb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:13.957667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64ade0-81f9-403d-a1a1-06f8d7c6a9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:13.963951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ade0-81f9-403d-a1a1-06f8d7c6a9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:13.964330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ade0-81f9-403d-a1a1-06f8d7c6a9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:13.964687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64ade0-81f9-403d-a1a1-06f8d7c6a9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:13.965564 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64ade0-81f9-403d-a1a1-06f8d7c6a9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:13.973163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a64ade0-81f9-403d-a1a1-06f8d7c6a9b3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:14.017108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 919/1852] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:14.018502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64ade0-81f9-403d-a1a1-06f8d7c6a9b3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:14.033382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64ade0-81f9-403d-a1a1-06f8d7c6a9b3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:14.041118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df327c-a060-4c1f-a8eb-48dc241263b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:14.042384 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df327c-a060-4c1f-a8eb-48dc241263b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:14.042749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df327c-a060-4c1f-a8eb-48dc241263b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:14.044332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df327c-a060-4c1f-a8eb-48dc241263b4 admin admin] PATH_INFO: `/v3/projects/37a67d8c857f4b58a227cc39dcdea404/users/3e6f478548c94ceea676aa4ab8ad6356/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:14.091086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df327c-a060-4c1f-a8eb-48dc241263b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a67d8c857f4b58a227cc39dcdea404, user_id=3e6f478548c94ceea676aa4ab8ad6356, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:14.097859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df327c-a060-4c1f-a8eb-48dc241263b4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:14.112062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:14.135139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 920/1853] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:14 2022] PUT /identity/v3/projects/37a67d8c857f4b58a227cc39dcdea404/users/3e6f478548c94ceea676aa4ab8ad6356/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:14.140443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2cc1-6822-4aaf-a18b-0ee2c0b095a6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:14.142072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2cc1-6822-4aaf-a18b-0ee2c0b095a6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:14.142727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2cc1-6822-4aaf-a18b-0ee2c0b095a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:14.174172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 934/1854] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:14.179154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4185d-f1db-4c9d-b4f2-08dab336a852 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:14.192985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2d1a2cc1-6822-4aaf-a18b-0ee2c0b095a6 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:14.195772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2d1a2cc1-6822-4aaf-a18b-0ee2c0b095a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:14.218832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4185d-f1db-4c9d-b4f2-08dab336a852 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:14.507114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:14.508567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:14.510802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:14.516720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 921/1855] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:31:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:14.516947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] PATH_INFO: `/v3/projects/c723600975514a77a647b0953eb7ad8c/users/f937c6752683410aba1570ea12c53377/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:14.519160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a9693-8363-4977-924f-8c5182c33fc6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:14.519639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a9693-8363-4977-924f-8c5182c33fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:14.520247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a9693-8363-4977-924f-8c5182c33fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:14.552927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:14.621542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c723600975514a77a647b0953eb7ad8c, user_id=f937c6752683410aba1570ea12c53377, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:14.631176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:14.647159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b18a9693-8363-4977-924f-8c5182c33fc6 None None] MFA Rules not processed for user `3e6f478548c94ceea676aa4ab8ad6356`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:14.650521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b18a9693-8363-4977-924f-8c5182c33fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:14.680401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 935/1856] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:14 2022] PUT /identity/v3/projects/c723600975514a77a647b0953eb7ad8c/users/f937c6752683410aba1570ea12c53377/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:14.683601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:14.690569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:14.898269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bcdd0-c579-4e01-a89e-1f01a2b2fba3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:14.921267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981d788-9951-4047-b764-62345c9603c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:14.922209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d788-9951-4047-b764-62345c9603c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:14.922670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d788-9951-4047-b764-62345c9603c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:14.923058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981d788-9951-4047-b764-62345c9603c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:14.948689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 922/1857] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:14.950422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a9693-8363-4977-924f-8c5182c33fc6 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:14.978520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a9693-8363-4977-924f-8c5182c33fc6 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:15.107063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981d788-9951-4047-b764-62345c9603c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:15.124724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981d788-9951-4047-b764-62345c9603c0 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:15.147886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 936/1858] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:15.150887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981d788-9951-4047-b764-62345c9603c0 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:15.152230 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ac0b9-a54a-4eb3-93d3-b6798ec4fe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:15.153257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ac0b9-a54a-4eb3-93d3-b6798ec4fe7a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:15.155720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ac0b9-a54a-4eb3-93d3-b6798ec4fe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:15.156109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ac0b9-a54a-4eb3-93d3-b6798ec4fe7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:15.158810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ac0b9-a54a-4eb3-93d3-b6798ec4fe7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:15.172307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ac0b9-a54a-4eb3-93d3-b6798ec4fe7a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:15.182577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981d788-9951-4047-b764-62345c9603c0 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:15.196311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 923/1859] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:15.203958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ac0b9-a54a-4eb3-93d3-b6798ec4fe7a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:15.205031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afd8fd-717b-40d4-a779-3f593506dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:15.206148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afd8fd-717b-40d4-a779-3f593506dd8f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:15.206460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afd8fd-717b-40d4-a779-3f593506dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:15.210632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afd8fd-717b-40d4-a779-3f593506dd8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:15.211485 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afd8fd-717b-40d4-a779-3f593506dd8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:15.223470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afd8fd-717b-40d4-a779-3f593506dd8f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:15.287713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ac0b9-a54a-4eb3-93d3-b6798ec4fe7a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:15.308501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d90928-2c50-4404-b486-54727c5fc672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:15.312607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d90928-2c50-4404-b486-54727c5fc672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:15.312953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d90928-2c50-4404-b486-54727c5fc672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:15.316457 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d90928-2c50-4404-b486-54727c5fc672 admin admin] PATH_INFO: `/v3/projects/c723600975514a77a647b0953eb7ad8c/users/f937c6752683410aba1570ea12c53377/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:15.326133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d90928-2c50-4404-b486-54727c5fc672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c723600975514a77a647b0953eb7ad8c, user_id=f937c6752683410aba1570ea12c53377, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:15.342450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d90928-2c50-4404-b486-54727c5fc672 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:15.374037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 937/1860] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:15 2022] POST /identity/v3/projects => generated 357 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:15.375088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afd8fd-717b-40d4-a779-3f593506dd8f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:15.385097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 924/1861] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:15 2022] PUT /identity/v3/projects/c723600975514a77a647b0953eb7ad8c/users/f937c6752683410aba1570ea12c53377/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:15.396641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2efef-3301-4453-bedb-2baf6c302da4 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:15.397152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2efef-3301-4453-bedb-2baf6c302da4 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:15.397339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2efef-3301-4453-bedb-2baf6c302da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:15.416102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:15.516916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8db2efef-3301-4453-bedb-2baf6c302da4 None None] MFA Rules not processed for user `f937c6752683410aba1570ea12c53377`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:15.554945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8db2efef-3301-4453-bedb-2baf6c302da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:15.631735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afd8fd-717b-40d4-a779-3f593506dd8f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:15.648852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:15.649808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:15.650612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:15.651044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:15.651975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:15.662373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:15.778347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 938/1862] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:15 2022] POST /identity/v3/users => generated 285 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:15.780087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:15.790053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:15.878715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:15.911900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 925/1863] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:15.922612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2efef-3301-4453-bedb-2baf6c302da4 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:15.933139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8db2efef-3301-4453-bedb-2baf6c302da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:16.000653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2efef-3301-4453-bedb-2baf6c302da4 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:16.019527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:16.020612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:16.021117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:16.021703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] PATH_INFO: `/v3/users/9697987d044249fba621cb6e2b787c54` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:16.030033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] RBAC: Authorizing `identity:delete_user(user_id=9697987d044249fba621cb6e2b787c54)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:16.036543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:16.096663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27808a-44d3-4a52-9d8d-7112fbaf0dd7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:16.117740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d51-f6dd-4da3-86fc-7a6086ca93d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:16.119825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d51-f6dd-4da3-86fc-7a6086ca93d7 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:16.120792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d51-f6dd-4da3-86fc-7a6086ca93d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:16.121240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d51-f6dd-4da3-86fc-7a6086ca93d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:16.162333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925d51-f6dd-4da3-86fc-7a6086ca93d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:16.171538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925d51-f6dd-4da3-86fc-7a6086ca93d7 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:16.236820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 939/1864] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:16.238563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d51-f6dd-4da3-86fc-7a6086ca93d7 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:16.250906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9697987d044249fba621cb6e2b787c54'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:16.254218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9697987d044249fba621cb6e2b787c54'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:16.255627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9697987d044249fba621cb6e2b787c54'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:16.288841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9697987d044249fba621cb6e2b787c54'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:16.292919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d51-f6dd-4da3-86fc-7a6086ca93d7 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:16.312481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:16.313519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:16.313894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:16.314332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:16.315133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:16.317700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:16.353590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8ef618b3-9707-4da0-953d-389c4edb95db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9697987d044249fba621cb6e2b787c54'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:16.366798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 940/1865] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:16.366798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:16.374115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:16.422126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 926/1866] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:15 2022] DELETE /identity/v3/users/9697987d044249fba621cb6e2b787c54 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:16.425859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83924aa0-935d-4e01-988f-df8dbb679375 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:16.426774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83924aa0-935d-4e01-988f-df8dbb679375 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:16.426897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83924aa0-935d-4e01-988f-df8dbb679375 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:16.429715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 927/1867] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:31:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:31:16.445947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83924aa0-935d-4e01-988f-df8dbb679375 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:16.462869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-83924aa0-935d-4e01-988f-df8dbb679375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:16.575591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b65fa6-76fa-4068-9bcd-701edbd15287 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:16.663654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:16.729236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:16.731501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:16.731796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:16.732085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:16.743656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:16.891181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83924aa0-935d-4e01-988f-df8dbb679375 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:16.905292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:16.907943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:16.909023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:16.909970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] PATH_INFO: `/v3/projects/b9009bc8238f44f0ba64769c8bef11d6` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:16.912827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9009bc8238f44f0ba64769c8bef11d6)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:16.921578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:16.967458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:16.968991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:16.992712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 941/1868] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:16.993700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:17.003773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:17.024309 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85bb90-1acc-442d-9efb-1f70fdb06ee5 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:17.059498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9009bc8238f44f0ba64769c8bef11d6', 'request_id': 'req-2b01774c-de16-43e8-8df0-3b85da60430f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.069639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9009bc8238f44f0ba64769c8bef11d6', 'request_id': 'req-2b01774c-de16-43e8-8df0-3b85da60430f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.079763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9009bc8238f44f0ba64769c8bef11d6', 'request_id': 'req-2b01774c-de16-43e8-8df0-3b85da60430f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.110389 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] The token cache is being invalidate because project b9009bc8238f44f0ba64769c8bef11d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:17.111140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.115551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2b01774c-de16-43e8-8df0-3b85da60430f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.118996 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 928/1869] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:16 2022] DELETE /identity/v3/projects/b9009bc8238f44f0ba64769c8bef11d6 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:17.129908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad434b-d556-43f9-a29b-7548770fa856 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:17.129908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad434b-d556-43f9-a29b-7548770fa856 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:17.130578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad434b-d556-43f9-a29b-7548770fa856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:17.158497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:17.203308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:17.204017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:17.204308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:17.204593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] PATH_INFO: `/v3/projects/d851da34bbd44716a65fbb0cafdf65d4/users/6462f65a0cfc41c782669fd6c4804e3e/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:17.206022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-79ad434b-d556-43f9-a29b-7548770fa856 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:17.214887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-79ad434b-d556-43f9-a29b-7548770fa856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:17.269444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d851da34bbd44716a65fbb0cafdf65d4, user_id=6462f65a0cfc41c782669fd6c4804e3e, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:17.273278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:17.298113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 929/1870] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:17.312721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d988d8-a1ec-4ec2-ab00-d91f87129796 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:17.314290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 942/1871] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:16 2022] PUT /identity/v3/projects/d851da34bbd44716a65fbb0cafdf65d4/users/6462f65a0cfc41c782669fd6c4804e3e/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:17.315290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:17.324004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:17.324748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d988d8-a1ec-4ec2-ab00-d91f87129796 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:17.330925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d988d8-a1ec-4ec2-ab00-d91f87129796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:17.512651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-25d988d8-a1ec-4ec2-ab00-d91f87129796 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:17.519979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-25d988d8-a1ec-4ec2-ab00-d91f87129796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:17.536372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc80d7e-57de-4899-b472-890d11f07422 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:17.551749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:17.551749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:17.552069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:17.552288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] PATH_INFO: `/v3/users/d7b8633fa8e042ca9122257aac50251e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:17.557505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7b8633fa8e042ca9122257aac50251e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:17.569167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:17.609335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:17.712075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7b8633fa8e042ca9122257aac50251e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.759521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7b8633fa8e042ca9122257aac50251e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.762246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7b8633fa8e042ca9122257aac50251e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.783964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7b8633fa8e042ca9122257aac50251e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.803256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7b8633fa8e042ca9122257aac50251e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:17.818938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 943/1872] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:17 2022] DELETE /identity/v3/users/d7b8633fa8e042ca9122257aac50251e => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:17.819953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:17.821577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:17.935813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 930/1873] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:31:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 637 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:17.938928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d988d8-a1ec-4ec2-ab00-d91f87129796 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:17.947883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-25d988d8-a1ec-4ec2-ab00-d91f87129796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:18.021538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78d5f9-ef50-439f-be64-461a06166eb3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:18.032525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c343f8d-26f4-47bf-b559-967c6dfb5989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:18.034271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343f8d-26f4-47bf-b559-967c6dfb5989 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:18.034637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343f8d-26f4-47bf-b559-967c6dfb5989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:18.035091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343f8d-26f4-47bf-b559-967c6dfb5989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:18.035853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c343f8d-26f4-47bf-b559-967c6dfb5989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:18.038089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c343f8d-26f4-47bf-b559-967c6dfb5989 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:18.102717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 944/1874] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:18.106654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c343f8d-26f4-47bf-b559-967c6dfb5989 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:18.155097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c343f8d-26f4-47bf-b559-967c6dfb5989 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:18.175218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:18.183085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:18.183721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:18.188948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] PATH_INFO: `/v3/projects/d851da34bbd44716a65fbb0cafdf65d4/users/6462f65a0cfc41c782669fd6c4804e3e/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:18.196588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d851da34bbd44716a65fbb0cafdf65d4, user_id=6462f65a0cfc41c782669fd6c4804e3e, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:18.206354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:18.232385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 945/1875] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:18 2022] PUT /identity/v3/projects/d851da34bbd44716a65fbb0cafdf65d4/users/6462f65a0cfc41c782669fd6c4804e3e/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:18.258763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:18.265132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:18.268085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d988d8-a1ec-4ec2-ab00-d91f87129796 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:18.396956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e17085-a16a-494a-9cc8-8bc2972c96cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:18.399727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e17085-a16a-494a-9cc8-8bc2972c96cc service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:18.400296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e17085-a16a-494a-9cc8-8bc2972c96cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:18.400686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e17085-a16a-494a-9cc8-8bc2972c96cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:18.405812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d3e17085-a16a-494a-9cc8-8bc2972c96cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:18.590144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634bdef-64ac-4440-95b2-23ae5028d4dc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:18.605515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:18.606441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:18.607983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:18.610586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:18.616089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:18.628246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:18.628943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e17085-a16a-494a-9cc8-8bc2972c96cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:18.631165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e17085-a16a-494a-9cc8-8bc2972c96cc service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:18.640733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:18.645143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:18.657292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 931/1876] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:18.659179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e8a2e-c5e9-415a-bb42-64aa5f64bf2d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:18.659666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e8a2e-c5e9-415a-bb42-64aa5f64bf2d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:18.660100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e8a2e-c5e9-415a-bb42-64aa5f64bf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:18.672005 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 946/1877] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:18.886704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-668e8a2e-c5e9-415a-bb42-64aa5f64bf2d None None] MFA Rules not processed for user `6462f65a0cfc41c782669fd6c4804e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:18.889183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-668e8a2e-c5e9-415a-bb42-64aa5f64bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:19.118428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 932/1878] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:18 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:19.125519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:19.130483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:19.267227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5ff56-55dc-4a47-b31d-7b1cbee1ae5b service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:19.280442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54288f22-c9bf-4d3c-83e4-68ddd91fd740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:19.287496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54288f22-c9bf-4d3c-83e4-68ddd91fd740 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:19.289063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54288f22-c9bf-4d3c-83e4-68ddd91fd740 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:19.289729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54288f22-c9bf-4d3c-83e4-68ddd91fd740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:19.318187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54288f22-c9bf-4d3c-83e4-68ddd91fd740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:19.321171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54288f22-c9bf-4d3c-83e4-68ddd91fd740 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:19.377681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 947/1879] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:19.500057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e8a2e-c5e9-415a-bb42-64aa5f64bf2d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:19.502536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-668e8a2e-c5e9-415a-bb42-64aa5f64bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:19.551186 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e8a2e-c5e9-415a-bb42-64aa5f64bf2d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:19.572105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:19.573518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:19.574246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:19.575144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] PATH_INFO: `/v3/projects/9b69262fc60a4b6a926e0776a50c71ac` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:19.584792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b69262fc60a4b6a926e0776a50c71ac)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:19.593159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:19.627932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54288f22-c9bf-4d3c-83e4-68ddd91fd740 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:19.657032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54288f22-c9bf-4d3c-83e4-68ddd91fd740 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:19.672058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:19.674149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:19.674628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:19.675007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:19.677935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:19.780257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b69262fc60a4b6a926e0776a50c71ac', 'request_id': 'req-de5314f1-aded-4244-9ee6-b5ae1f335790'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:19.808922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b69262fc60a4b6a926e0776a50c71ac', 'request_id': 'req-de5314f1-aded-4244-9ee6-b5ae1f335790'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:19.837525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b69262fc60a4b6a926e0776a50c71ac', 'request_id': 'req-de5314f1-aded-4244-9ee6-b5ae1f335790'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:19.881069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] The token cache is being invalidate because project 9b69262fc60a4b6a926e0776a50c71ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:19.881069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:19.889105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:19.901761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 933/1880] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:19 2022] DELETE /identity/v3/projects/9b69262fc60a4b6a926e0776a50c71ac => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:19.903839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:19.909004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:19.933710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:19.939915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:19.972586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 948/1881] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:31:20.131616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5314f1-aded-4244-9ee6-b5ae1f335790 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:20.141229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:20.142006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.142291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.142585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] PATH_INFO: `/v3/users/e69662ac9b8c48dd8c265b0c95478203` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.143953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69662ac9b8c48dd8c265b0c95478203)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:20.146928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:20.177290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:20.228202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:20.230010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:20.265853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9b78d-d628-4a23-a5b6-443e8373e412 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:20.286625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:20.287962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.288165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.288792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] PATH_INFO: `/v3/projects/66b13b68edfb48609eee2d1bb2867f37` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.295780 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b13b68edfb48609eee2d1bb2867f37)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:20.304874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:20.330075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69662ac9b8c48dd8c265b0c95478203'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.330971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69662ac9b8c48dd8c265b0c95478203'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.336834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69662ac9b8c48dd8c265b0c95478203'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.354533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69662ac9b8c48dd8c265b0c95478203'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.379471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69662ac9b8c48dd8c265b0c95478203'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.399007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 934/1882] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:19 2022] DELETE /identity/v3/users/e69662ac9b8c48dd8c265b0c95478203 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:20.400197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:20.401701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:20.411666 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b13b68edfb48609eee2d1bb2867f37', 'request_id': 'req-66301f34-2ac4-4d01-a6e6-65245af32232'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.423414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b13b68edfb48609eee2d1bb2867f37', 'request_id': 'req-66301f34-2ac4-4d01-a6e6-65245af32232'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.440155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b13b68edfb48609eee2d1bb2867f37', 'request_id': 'req-66301f34-2ac4-4d01-a6e6-65245af32232'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.488626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] The token cache is being invalidate because project 66b13b68edfb48609eee2d1bb2867f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:20.489228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.491423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-66301f34-2ac4-4d01-a6e6-65245af32232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.501877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 949/1883] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:20 2022] DELETE /identity/v3/projects/66b13b68edfb48609eee2d1bb2867f37 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:20.508484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2275-edeb-4d3e-8605-d37280faf6b5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.509314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2275-edeb-4d3e-8605-d37280faf6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.509796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2275-edeb-4d3e-8605-d37280faf6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.521831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf9350-6bd0-487d-ac7c-8f3c579dfdc6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:20.528151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:20.529549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.529944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.530432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] PATH_INFO: `/v3/projects/39f7be9679a84619ad235ec1b0f83f93` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.533546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39f7be9679a84619ad235ec1b0f83f93)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:20.536267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:20.568798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-647b2275-edeb-4d3e-8605-d37280faf6b5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:20.570881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-647b2275-edeb-4d3e-8605-d37280faf6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:20.590380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 950/1884] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:20.600445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb6a6-2360-483d-b0e4-4ec7e5955b94 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.600912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb6a6-2360-483d-b0e4-4ec7e5955b94 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.601285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fb6a6-2360-483d-b0e4-4ec7e5955b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.628638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ee9fb6a6-2360-483d-b0e4-4ec7e5955b94 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:20.629948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ee9fb6a6-2360-483d-b0e4-4ec7e5955b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:20.633778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39f7be9679a84619ad235ec1b0f83f93', 'request_id': 'req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.644127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39f7be9679a84619ad235ec1b0f83f93', 'request_id': 'req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.652255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:20.653832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39f7be9679a84619ad235ec1b0f83f93', 'request_id': 'req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.683086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] The token cache is being invalidate because project 39f7be9679a84619ad235ec1b0f83f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:20.683398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.684625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.687827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 935/1885] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:20 2022] DELETE /identity/v3/projects/39f7be9679a84619ad235ec1b0f83f93 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:20.692150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:20.698820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:20.726711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:20.757955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 951/1886] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:20.762935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fb6a6-2360-483d-b0e4-4ec7e5955b94 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:20.776166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9fb6a6-2360-483d-b0e4-4ec7e5955b94 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:20.782815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:20.783848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.784299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.784755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.786548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:20.787038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:20.789074 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:20.799351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092742f2-53d5-4cf5-a1f9-f5f4dd1ccd80 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:20.804437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:20.808722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.808946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 952/1887] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:20.809052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.809052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] PATH_INFO: `/v3/users/1dfe0ef9b5f049f1b7861818325782b4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.810210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dfe0ef9b5f049f1b7861818325782b4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:20.810361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:20.813356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:20.822208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8ce32-5a23-48a8-977d-470babc76ee2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:20.828681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:20.829323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.829725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.830015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.830742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:20.832945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:20.872618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 953/1888] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:20 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:20.874334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:20.877281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:20.890423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dfe0ef9b5f049f1b7861818325782b4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.891636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dfe0ef9b5f049f1b7861818325782b4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.892816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dfe0ef9b5f049f1b7861818325782b4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.914276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dfe0ef9b5f049f1b7861818325782b4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.945582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dfe0ef9b5f049f1b7861818325782b4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:20.965225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357670f-c9e7-4d80-bb31-7b4aa58f73f8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:20.969997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 936/1889] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:20 2022] DELETE /identity/v3/users/1dfe0ef9b5f049f1b7861818325782b4 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:20.970983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:20.973720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:20.976800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:20.977657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:20.978204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:20.979001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] PATH_INFO: `/v3/projects/f1cd2491d6c44a178ed7ac5336d6e269` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:20.981387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1cd2491d6c44a178ed7ac5336d6e269)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:20.986319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.073443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1cd2491d6c44a178ed7ac5336d6e269', 'request_id': 'req-90ea4f59-1d34-4309-935b-864d23569c51'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.092664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f404-ebc8-42bd-ae97-40a8075f0170 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.099699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1cd2491d6c44a178ed7ac5336d6e269', 'request_id': 'req-90ea4f59-1d34-4309-935b-864d23569c51'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.120236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1cd2491d6c44a178ed7ac5336d6e269', 'request_id': 'req-90ea4f59-1d34-4309-935b-864d23569c51'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.139688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] The token cache is being invalidate because project f1cd2491d6c44a178ed7ac5336d6e269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:21.140881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.141782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-90ea4f59-1d34-4309-935b-864d23569c51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.149018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 954/1890] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:20 2022] DELETE /identity/v3/projects/f1cd2491d6c44a178ed7ac5336d6e269 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:21.152052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83846157-2472-4e9a-b617-3fb5f49da856 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.152327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83846157-2472-4e9a-b617-3fb5f49da856 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.152623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83846157-2472-4e9a-b617-3fb5f49da856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.188185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:21.192649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-83846157-2472-4e9a-b617-3fb5f49da856 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:21.194169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-83846157-2472-4e9a-b617-3fb5f49da856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:21.205691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:21.272231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:21.273356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.273931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.274421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.276509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:21.279009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.320197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 937/1891] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:20 2022] POST /identity/v3/users => generated 287 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:21.326686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.331014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:21.359306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fd24b-74e0-4a7e-80d8-fbf97f9cf907 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.379718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 955/1892] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:21.380844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617935e2-a5b7-421b-85e2-a505b9ce7348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:21.380844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617935e2-a5b7-421b-85e2-a505b9ce7348 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.380844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617935e2-a5b7-421b-85e2-a505b9ce7348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.380844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617935e2-a5b7-421b-85e2-a505b9ce7348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.381372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617935e2-a5b7-421b-85e2-a505b9ce7348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:21.381372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617935e2-a5b7-421b-85e2-a505b9ce7348 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.390201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df811d-e8ef-49e2-a593-4a95508ad7c2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.390751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df811d-e8ef-49e2-a593-4a95508ad7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.391170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df811d-e8ef-49e2-a593-4a95508ad7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.415783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 938/1893] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:21.417747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617935e2-a5b7-421b-85e2-a505b9ce7348 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.432819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-18df811d-e8ef-49e2-a593-4a95508ad7c2 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:21.435173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-18df811d-e8ef-49e2-a593-4a95508ad7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:21.439325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617935e2-a5b7-421b-85e2-a505b9ce7348 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.447936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ad603-1f55-4dbd-bdb4-302a4121f48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:21.449020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ad603-1f55-4dbd-bdb4-302a4121f48d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.449561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ad603-1f55-4dbd-bdb4-302a4121f48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.450037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ad603-1f55-4dbd-bdb4-302a4121f48d admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.451370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ad603-1f55-4dbd-bdb4-302a4121f48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:21.452027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 956/1894] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:21.455273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ad603-1f55-4dbd-bdb4-302a4121f48d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.458957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df811d-e8ef-49e2-a593-4a95508ad7c2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.475554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df811d-e8ef-49e2-a593-4a95508ad7c2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.484166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 939/1895] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:21.486816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ad603-1f55-4dbd-bdb4-302a4121f48d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.487333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:21.488243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.489197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.489514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.490216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:21.490529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:21.492247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.514588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ad603-1f55-4dbd-bdb4-302a4121f48d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.521663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 957/1896] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:21.523241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.525023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6874-e27c-46fc-96e5-9d8ed676271f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:21.526241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6874-e27c-46fc-96e5-9d8ed676271f admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.527001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6874-e27c-46fc-96e5-9d8ed676271f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.528054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6874-e27c-46fc-96e5-9d8ed676271f admin admin] PATH_INFO: `/v3/projects/35d3c249756a4c969492e61ea3f6029c/users/01fa5689c515459686aafa27872d1c7a/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.542261 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e45bc-d3e0-49b8-9578-7053bbcd277b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.553416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb308160-9eee-40f9-b038-b62121b682a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:21.554582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb308160-9eee-40f9-b038-b62121b682a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.555154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb308160-9eee-40f9-b038-b62121b682a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.555685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb308160-9eee-40f9-b038-b62121b682a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.557257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb308160-9eee-40f9-b038-b62121b682a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:21.563711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6874-e27c-46fc-96e5-9d8ed676271f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35d3c249756a4c969492e61ea3f6029c, user_id=01fa5689c515459686aafa27872d1c7a, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:21.564678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb308160-9eee-40f9-b038-b62121b682a6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.567104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6874-e27c-46fc-96e5-9d8ed676271f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.592998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 940/1897] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:21 2022] PUT /identity/v3/projects/35d3c249756a4c969492e61ea3f6029c/users/01fa5689c515459686aafa27872d1c7a/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:21.601152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597836-c958-438c-800f-caf2760955aa None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.601490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597836-c958-438c-800f-caf2760955aa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.601825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92597836-c958-438c-800f-caf2760955aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.645783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 958/1898] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:21 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:21.645963 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-92597836-c958-438c-800f-caf2760955aa None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:21.646649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-92597836-c958-438c-800f-caf2760955aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:21.648123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb308160-9eee-40f9-b038-b62121b682a6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.764060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb308160-9eee-40f9-b038-b62121b682a6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.772257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:21.773118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.773664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.774146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.775280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:21.778769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.779900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 941/1899] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:21.780767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92597836-c958-438c-800f-caf2760955aa None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.782444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-92597836-c958-438c-800f-caf2760955aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:21.813968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 959/1900] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:21 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:21.815290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92597836-c958-438c-800f-caf2760955aa None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.815938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.819448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:21.824332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:21.825511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:21.827116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:21.827653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] PATH_INFO: `/v3/projects/d2f57026c00a449e98ece599d95c2336` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:21.829717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f57026c00a449e98ece599d95c2336)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:21.833004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:21.920873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f57026c00a449e98ece599d95c2336', 'request_id': 'req-1f9552e9-b7cd-4ccf-ba89-799398a41889'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.928717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec88881-b147-4b94-a3e2-89c0723c4e27 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:21.933224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f57026c00a449e98ece599d95c2336', 'request_id': 'req-1f9552e9-b7cd-4ccf-ba89-799398a41889'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.944860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f57026c00a449e98ece599d95c2336', 'request_id': 'req-1f9552e9-b7cd-4ccf-ba89-799398a41889'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.945225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:21.975292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] The token cache is being invalidate because project d2f57026c00a449e98ece599d95c2336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:21.975594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.976737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:21.980038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 942/1901] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:21 2022] DELETE /identity/v3/projects/d2f57026c00a449e98ece599d95c2336 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:21.982071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:21.985318 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:22.002660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:22.034142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f647e39-1886-4c4b-adbe-401595e9ad2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.035035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647e39-1886-4c4b-adbe-401595e9ad2b service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.035700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647e39-1886-4c4b-adbe-401595e9ad2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.036166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f647e39-1886-4c4b-adbe-401595e9ad2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.039621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0f647e39-1886-4c4b-adbe-401595e9ad2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:22.127550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9552e9-b7cd-4ccf-ba89-799398a41889 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:22.132738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f647e39-1886-4c4b-adbe-401595e9ad2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.133737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f4504-a151-4ec7-ada4-82cee6b1e741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.134653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f4504-a151-4ec7-ada4-82cee6b1e741 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.134836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f647e39-1886-4c4b-adbe-401595e9ad2b service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:22.135139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f4504-a151-4ec7-ada4-82cee6b1e741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.135723 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f4504-a151-4ec7-ada4-82cee6b1e741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.136599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f4504-a151-4ec7-ada4-82cee6b1e741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.138712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f4504-a151-4ec7-ada4-82cee6b1e741 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:22.150493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 960/1902] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:22.150493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d66a4b-2091-4786-8801-79e0eea20d83 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.152460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d66a4b-2091-4786-8801-79e0eea20d83 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.152460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d66a4b-2091-4786-8801-79e0eea20d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.166722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 943/1903] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:22.171992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f4504-a151-4ec7-ada4-82cee6b1e741 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:22.186449 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f4504-a151-4ec7-ada4-82cee6b1e741 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:22.188904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-00d66a4b-2091-4786-8801-79e0eea20d83 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:22.191192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-00d66a4b-2091-4786-8801-79e0eea20d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:22.193624 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76007670-a74d-4e7e-b805-48ac3abecf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.194499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76007670-a74d-4e7e-b805-48ac3abecf1e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.195216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76007670-a74d-4e7e-b805-48ac3abecf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.195216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76007670-a74d-4e7e-b805-48ac3abecf1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.196080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76007670-a74d-4e7e-b805-48ac3abecf1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.198236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76007670-a74d-4e7e-b805-48ac3abecf1e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:22.213575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 961/1904] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:22.213843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 944/1905] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:31:22.219825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d66a4b-2091-4786-8801-79e0eea20d83 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:22.220648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee6c0-9fba-4997-a71e-5d578e17d020 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.221178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee6c0-9fba-4997-a71e-5d578e17d020 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.221678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ee6c0-9fba-4997-a71e-5d578e17d020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.245538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d66a4b-2091-4786-8801-79e0eea20d83 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:22.249871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3d8ee6c0-9fba-4997-a71e-5d578e17d020 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:22.251808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3d8ee6c0-9fba-4997-a71e-5d578e17d020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:22.256080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccaffe-3d74-4a48-80ef-f819e308c698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.257694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccaffe-3d74-4a48-80ef-f819e308c698 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.258211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccaffe-3d74-4a48-80ef-f819e308c698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.258943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccaffe-3d74-4a48-80ef-f819e308c698 admin admin] PATH_INFO: `/v3/projects/7f4aecbea25b4923b878f787b425f102/users/c53439fafa0a40ae8ae580e3916e3392/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.268470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 945/1906] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:22.276331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ee6c0-9fba-4997-a71e-5d578e17d020 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:22.293628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ee6c0-9fba-4997-a71e-5d578e17d020 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:22.304141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccaffe-3d74-4a48-80ef-f819e308c698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4aecbea25b4923b878f787b425f102, user_id=c53439fafa0a40ae8ae580e3916e3392, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.308281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.309378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.309849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.311120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.312444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:22.312999 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.316552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccaffe-3d74-4a48-80ef-f819e308c698 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:22.316829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:22.354306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 962/1907] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:22 2022] PUT /identity/v3/projects/7f4aecbea25b4923b878f787b425f102/users/c53439fafa0a40ae8ae580e3916e3392/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:22.370226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a319-8fa2-4277-9ff5-10f7395f1996 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.370863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a319-8fa2-4277-9ff5-10f7395f1996 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.371719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6a319-8fa2-4277-9ff5-10f7395f1996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.390621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 946/1908] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:22.392382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:22.438397 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20a8b1-fece-4467-998a-3e7a2edd79d6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:22.444051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1ee6a319-8fa2-4277-9ff5-10f7395f1996 None None] MFA Rules not processed for user `c53439fafa0a40ae8ae580e3916e3392`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:22.446898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1ee6a319-8fa2-4277-9ff5-10f7395f1996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:22.629754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a18ab-3329-4025-a0ff-4a8e5b687eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.631348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a18ab-3329-4025-a0ff-4a8e5b687eba admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.631981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a18ab-3329-4025-a0ff-4a8e5b687eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.632458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a18ab-3329-4025-a0ff-4a8e5b687eba admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.633523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a18ab-3329-4025-a0ff-4a8e5b687eba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.636486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a18ab-3329-4025-a0ff-4a8e5b687eba admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:22.710655 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 947/1909] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:22 2022] POST /identity/v3/projects => generated 363 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:22.718561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a18ab-3329-4025-a0ff-4a8e5b687eba admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:22.726626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 963/1910] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:22.734630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6a319-8fa2-4277-9ff5-10f7395f1996 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:22.785394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6a319-8fa2-4277-9ff5-10f7395f1996 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:22.835767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:22.866260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:22.879255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a18ab-3329-4025-a0ff-4a8e5b687eba admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:22.885178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3f33-76d8-48aa-9ea4-c7045b66fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.886373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3f33-76d8-48aa-9ea4-c7045b66fc9f admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.892085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3f33-76d8-48aa-9ea4-c7045b66fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.892085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3f33-76d8-48aa-9ea4-c7045b66fc9f admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.892085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b3f33-76d8-48aa-9ea4-c7045b66fc9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.892085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b3f33-76d8-48aa-9ea4-c7045b66fc9f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:22.938368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 948/1911] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:22 2022] POST /identity/v3/users => generated 282 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:22.945131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3f33-76d8-48aa-9ea4-c7045b66fc9f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:22.964716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1755e-5458-4288-98d0-3a741c8bebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.966154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1755e-5458-4288-98d0-3a741c8bebd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.968153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1755e-5458-4288-98d0-3a741c8bebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.968726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1755e-5458-4288-98d0-3a741c8bebd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.970634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1755e-5458-4288-98d0-3a741c8bebd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.977491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1755e-5458-4288-98d0-3a741c8bebd1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:22.983139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3f33-76d8-48aa-9ea4-c7045b66fc9f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:22.990931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828e594-05f3-491d-bf24-2acbfa56bcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:22.991747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828e594-05f3-491d-bf24-2acbfa56bcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:22.992114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828e594-05f3-491d-bf24-2acbfa56bcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:22.992438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828e594-05f3-491d-bf24-2acbfa56bcb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:22.993260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828e594-05f3-491d-bf24-2acbfa56bcb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:22.995602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828e594-05f3-491d-bf24-2acbfa56bcb7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.015499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 949/1912] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:23.016917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828e594-05f3-491d-bf24-2acbfa56bcb7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.034227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828e594-05f3-491d-bf24-2acbfa56bcb7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.040674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35f1ab-5dd1-494e-ae5d-f5a4e37127c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.041386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35f1ab-5dd1-494e-ae5d-f5a4e37127c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.041724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35f1ab-5dd1-494e-ae5d-f5a4e37127c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.042060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35f1ab-5dd1-494e-ae5d-f5a4e37127c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.042872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35f1ab-5dd1-494e-ae5d-f5a4e37127c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.045212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35f1ab-5dd1-494e-ae5d-f5a4e37127c6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.071360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 950/1913] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:23.073087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35f1ab-5dd1-494e-ae5d-f5a4e37127c6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.073279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 964/1914] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:22 2022] POST /identity/v3/projects => generated 379 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:23.077961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1755e-5458-4288-98d0-3a741c8bebd1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.099067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35f1ab-5dd1-494e-ae5d-f5a4e37127c6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.233309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adfaee-ea98-48a2-9e6b-8da99fcc6b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.234336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adfaee-ea98-48a2-9e6b-8da99fcc6b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.234696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adfaee-ea98-48a2-9e6b-8da99fcc6b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.235052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adfaee-ea98-48a2-9e6b-8da99fcc6b85 admin admin] PATH_INFO: `/v3/projects/e2d99547a70949368d202ab42771e4e4/users/5a1e98f27b724b2a993a4ee96e9e024e/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.259785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adfaee-ea98-48a2-9e6b-8da99fcc6b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d99547a70949368d202ab42771e4e4, user_id=5a1e98f27b724b2a993a4ee96e9e024e, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.260277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1755e-5458-4288-98d0-3a741c8bebd1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.266486 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a8c47-e439-408e-82db-e525acc259a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.267483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a8c47-e439-408e-82db-e525acc259a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.267709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a8c47-e439-408e-82db-e525acc259a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.267971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adfaee-ea98-48a2-9e6b-8da99fcc6b85 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.268080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a8c47-e439-408e-82db-e525acc259a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.268646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a8c47-e439-408e-82db-e525acc259a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.270898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a8c47-e439-408e-82db-e525acc259a4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.289875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 951/1915] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:23 2022] PUT /identity/v3/projects/e2d99547a70949368d202ab42771e4e4/users/5a1e98f27b724b2a993a4ee96e9e024e/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:23.306232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952a825-433a-4fcb-ba6d-03879b04ca32 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.306739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952a825-433a-4fcb-ba6d-03879b04ca32 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.307246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952a825-433a-4fcb-ba6d-03879b04ca32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.327197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 965/1916] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:23 2022] POST /identity/v3/users => generated 296 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:23.339625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a8c47-e439-408e-82db-e525acc259a4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.360581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a8c47-e439-408e-82db-e525acc259a4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.364252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e952a825-433a-4fcb-ba6d-03879b04ca32 None None] MFA Rules not processed for user `5a1e98f27b724b2a993a4ee96e9e024e`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:23.366051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e952a825-433a-4fcb-ba6d-03879b04ca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:23.453572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5be409-7693-4480-8efe-06712ffc7867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.454424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5be409-7693-4480-8efe-06712ffc7867 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.454616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5be409-7693-4480-8efe-06712ffc7867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.454954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5be409-7693-4480-8efe-06712ffc7867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.455705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5be409-7693-4480-8efe-06712ffc7867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.457643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5be409-7693-4480-8efe-06712ffc7867 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.473323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 966/1917] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:23.474319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5be409-7693-4480-8efe-06712ffc7867 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.489097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5be409-7693-4480-8efe-06712ffc7867 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.494259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 952/1918] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:23.494510 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ab7b1-d755-4dda-b333-31e9b3a7344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.494872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab7b1-d755-4dda-b333-31e9b3a7344b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.495147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab7b1-d755-4dda-b333-31e9b3a7344b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.495425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ab7b1-d755-4dda-b333-31e9b3a7344b admin admin] PATH_INFO: `/v3/projects/ef1717ff2ccd4295b9d5194c692308af/users/f8bb173a4ffa47b3b621749c214b4d7f/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.501193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952a825-433a-4fcb-ba6d-03879b04ca32 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.512498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952a825-433a-4fcb-ba6d-03879b04ca32 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.518325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ab7b1-d755-4dda-b333-31e9b3a7344b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1717ff2ccd4295b9d5194c692308af, user_id=f8bb173a4ffa47b3b621749c214b4d7f, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.519214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f2af3-dab8-46a3-889e-5c9f93c7caa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.520103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f2af3-dab8-46a3-889e-5c9f93c7caa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.520389 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f2af3-dab8-46a3-889e-5c9f93c7caa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.520687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f2af3-dab8-46a3-889e-5c9f93c7caa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.521145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ab7b1-d755-4dda-b333-31e9b3a7344b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.521326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f2af3-dab8-46a3-889e-5c9f93c7caa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.523332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f2af3-dab8-46a3-889e-5c9f93c7caa3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.542459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 967/1919] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:23 2022] PUT /identity/v3/projects/ef1717ff2ccd4295b9d5194c692308af/users/f8bb173a4ffa47b3b621749c214b4d7f/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:23.548394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ab7b1-d755-4dda-b333-31e9b3a7344b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.560795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:23.576900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 953/1920] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:23 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:23.579183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f2af3-dab8-46a3-889e-5c9f93c7caa3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.611147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:23.672272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ab7b1-d755-4dda-b333-31e9b3a7344b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.768347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f2af3-dab8-46a3-889e-5c9f93c7caa3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.789648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbef36-455d-40a6-be63-e4302c0ae984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.802344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbef36-455d-40a6-be63-e4302c0ae984 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.802807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbef36-455d-40a6-be63-e4302c0ae984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.803175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbef36-455d-40a6-be63-e4302c0ae984 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.804336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbef36-455d-40a6-be63-e4302c0ae984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.812943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbef36-455d-40a6-be63-e4302c0ae984 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.891854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 954/1921] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:23 2022] POST /identity/v3/users => generated 288 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:23.893317 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e2e0b-317a-4630-83e7-8293213f1ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.894023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2e0b-317a-4630-83e7-8293213f1ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.894754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2e0b-317a-4630-83e7-8293213f1ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.895393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e2e0b-317a-4630-83e7-8293213f1ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.896736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e2e0b-317a-4630-83e7-8293213f1ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.898837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dd244-98ce-4433-acb2-e429ebcfdd3c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.900099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e2e0b-317a-4630-83e7-8293213f1ac8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:23.902775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dd244-98ce-4433-acb2-e429ebcfdd3c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.904847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dd244-98ce-4433-acb2-e429ebcfdd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.930414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 968/1922] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:23.930414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e2e0b-317a-4630-83e7-8293213f1ac8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:23.962742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e2e0b-317a-4630-83e7-8293213f1ac8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:23.985270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910270d1-589e-4077-88fe-ce05171d4fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:23.985270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910270d1-589e-4077-88fe-ce05171d4fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:23.985270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910270d1-589e-4077-88fe-ce05171d4fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:23.985270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910270d1-589e-4077-88fe-ce05171d4fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:23.987127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910270d1-589e-4077-88fe-ce05171d4fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:23.987127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910270d1-589e-4077-88fe-ce05171d4fe1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:24.000320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 969/1923] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:24.008402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910270d1-589e-4077-88fe-ce05171d4fe1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:24.027958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910270d1-589e-4077-88fe-ce05171d4fe1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:24.032441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1e5dd244-98ce-4433-acb2-e429ebcfdd3c None None] MFA Rules not processed for user `01fa5689c515459686aafa27872d1c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:24.035165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1e5dd244-98ce-4433-acb2-e429ebcfdd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:24.042554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ff7f4-aac2-458c-86c3-84c597ae1409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:24.043267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ff7f4-aac2-458c-86c3-84c597ae1409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:24.043557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ff7f4-aac2-458c-86c3-84c597ae1409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:24.043889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ff7f4-aac2-458c-86c3-84c597ae1409 admin admin] PATH_INFO: `/v3/projects/ef1717ff2ccd4295b9d5194c692308af/users/f8bb173a4ffa47b3b621749c214b4d7f/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:24.050872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ff7f4-aac2-458c-86c3-84c597ae1409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1717ff2ccd4295b9d5194c692308af, user_id=f8bb173a4ffa47b3b621749c214b4d7f, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:24.053747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ff7f4-aac2-458c-86c3-84c597ae1409 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:24.083677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 970/1924] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:24 2022] PUT /identity/v3/projects/ef1717ff2ccd4295b9d5194c692308af/users/f8bb173a4ffa47b3b621749c214b4d7f/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:24.084576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ff7f4-aac2-458c-86c3-84c597ae1409 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:24.288052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ff7f4-aac2-458c-86c3-84c597ae1409 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:24.299627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 955/1925] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:24.299627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d88bf8-b239-42c6-8942-4ff6bbd3fa7a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:24.299627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d88bf8-b239-42c6-8942-4ff6bbd3fa7a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:24.299627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d88bf8-b239-42c6-8942-4ff6bbd3fa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:24.303722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:24.304445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:24.304808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:24.305056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] PATH_INFO: `/v3/projects/2363bf836b9944fca276c28fb87f088c/users/e338101ac6944cd4b64f2eae7e9453fb/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:24.360785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2363bf836b9944fca276c28fb87f088c, user_id=e338101ac6944cd4b64f2eae7e9453fb, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:24.363915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:24.386350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 971/1926] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:24 2022] PUT /identity/v3/projects/2363bf836b9944fca276c28fb87f088c/users/e338101ac6944cd4b64f2eae7e9453fb/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:24.387669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:24.390594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:24.394261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f8d88bf8-b239-42c6-8942-4ff6bbd3fa7a None None] MFA Rules not processed for user `f8bb173a4ffa47b3b621749c214b4d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:24.397158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f8d88bf8-b239-42c6-8942-4ff6bbd3fa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:24.416728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:24.505653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f7372-2099-4e81-8cde-12a706c9a0fd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:24.517327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:24.519305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:24.519952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:24.520230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:24.533065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:24.583308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 956/1927] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:24.598749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d88bf8-b239-42c6-8942-4ff6bbd3fa7a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:24.603131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f8d88bf8-b239-42c6-8942-4ff6bbd3fa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:24.663850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:24.678846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:24.680742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:24.700453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 972/1928] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:24.703225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:24.782445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d88bf8-b239-42c6-8942-4ff6bbd3fa7a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:24.797640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14568ad7-cf3f-48b4-9de0-b146875a4920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:24.798302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568ad7-cf3f-48b4-9de0-b146875a4920 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:24.798577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568ad7-cf3f-48b4-9de0-b146875a4920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:24.798853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14568ad7-cf3f-48b4-9de0-b146875a4920 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:24.799522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14568ad7-cf3f-48b4-9de0-b146875a4920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:24.803654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14568ad7-cf3f-48b4-9de0-b146875a4920 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:24.845559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c069959-d913-49fd-9d6a-270aa30eefdc service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:24.853519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:24.854384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:24.854677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:24.854994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:24.855870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:24.859004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:24.883101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 973/1929] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:24.883101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:24.883540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 957/1930] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:24 2022] POST /identity/v3/projects => generated 371 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:24.883540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14568ad7-cf3f-48b4-9de0-b146875a4920 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:24.888114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:24.930019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f0cb-2283-468d-8e6e-340ff9c189c0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:25.036645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e042-e284-41f1-bad9-b964e38dd85f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:25.037398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e042-e284-41f1-bad9-b964e38dd85f service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.037753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e042-e284-41f1-bad9-b964e38dd85f service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.038056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e042-e284-41f1-bad9-b964e38dd85f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:25.052811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:25.055456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14568ad7-cf3f-48b4-9de0-b146875a4920 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:25.082674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071408-204f-4717-b571-59443b320e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:25.083539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071408-204f-4717-b571-59443b320e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.083818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071408-204f-4717-b571-59443b320e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.084103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46071408-204f-4717-b571-59443b320e35 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:25.084876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46071408-204f-4717-b571-59443b320e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:25.087114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46071408-204f-4717-b571-59443b320e35 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:25.115902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 958/1931] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:24 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:25.118607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071408-204f-4717-b571-59443b320e35 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:25.132192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46071408-204f-4717-b571-59443b320e35 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:25.138319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ec4f8-464a-4ca0-8959-0d03f1beb13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:25.139346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ec4f8-464a-4ca0-8959-0d03f1beb13c admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.139753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ec4f8-464a-4ca0-8959-0d03f1beb13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.140001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ec4f8-464a-4ca0-8959-0d03f1beb13c admin admin] PATH_INFO: `/v3/projects/2363bf836b9944fca276c28fb87f088c/users/e338101ac6944cd4b64f2eae7e9453fb/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:25.142258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ec4f8-464a-4ca0-8959-0d03f1beb13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2363bf836b9944fca276c28fb87f088c, user_id=e338101ac6944cd4b64f2eae7e9453fb, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:25.145311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ec4f8-464a-4ca0-8959-0d03f1beb13c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:25.168127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 959/1932] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:25 2022] PUT /identity/v3/projects/2363bf836b9944fca276c28fb87f088c/users/e338101ac6944cd4b64f2eae7e9453fb/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:25.168127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ec4f8-464a-4ca0-8959-0d03f1beb13c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:25.171955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78e042-e284-41f1-bad9-b964e38dd85f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:25.179883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78e042-e284-41f1-bad9-b964e38dd85f service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:25.202822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 974/1933] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:25.205302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aefc6d-be5e-4f5a-8c4e-5db07af95477 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.206024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aefc6d-be5e-4f5a-8c4e-5db07af95477 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.207050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aefc6d-be5e-4f5a-8c4e-5db07af95477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:25.261381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-75aefc6d-be5e-4f5a-8c4e-5db07af95477 None None] MFA Rules not processed for user `e338101ac6944cd4b64f2eae7e9453fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:25.262894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-75aefc6d-be5e-4f5a-8c4e-5db07af95477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:25.333993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ec4f8-464a-4ca0-8959-0d03f1beb13c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:25.343229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29164c-8281-45f2-9d74-23b842bcbc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:25.343870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29164c-8281-45f2-9d74-23b842bcbc75 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.344159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29164c-8281-45f2-9d74-23b842bcbc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.344449 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29164c-8281-45f2-9d74-23b842bcbc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:25.345141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29164c-8281-45f2-9d74-23b842bcbc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:25.346863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29164c-8281-45f2-9d74-23b842bcbc75 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:25.367056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 960/1934] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:25.371776 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29164c-8281-45f2-9d74-23b842bcbc75 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:25.392628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29164c-8281-45f2-9d74-23b842bcbc75 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:25.398668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984496e-63e9-4f0c-bd04-b6e6d0124542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:25.400155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984496e-63e9-4f0c-bd04-b6e6d0124542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.400442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984496e-63e9-4f0c-bd04-b6e6d0124542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.400772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984496e-63e9-4f0c-bd04-b6e6d0124542 admin admin] PATH_INFO: `/v3/projects/443b0cf1604e4b74ab4f4db3ddb37bd8/users/076943a35907479f8759c423ed4871d9/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:25.418246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:25.427281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984496e-63e9-4f0c-bd04-b6e6d0124542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=443b0cf1604e4b74ab4f4db3ddb37bd8, user_id=076943a35907479f8759c423ed4871d9, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:25.430144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984496e-63e9-4f0c-bd04-b6e6d0124542 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:25.456087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 961/1935] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:25 2022] PUT /identity/v3/projects/443b0cf1604e4b74ab4f4db3ddb37bd8/users/076943a35907479f8759c423ed4871d9/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:25.462617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984496e-63e9-4f0c-bd04-b6e6d0124542 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:25.539924 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 975/1936] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:25.575538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aefc6d-be5e-4f5a-8c4e-5db07af95477 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:25.610546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984496e-63e9-4f0c-bd04-b6e6d0124542 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:25.624993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8c397-ff66-492d-bfdc-802690486eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:25.625751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c397-ff66-492d-bfdc-802690486eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.626027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c397-ff66-492d-bfdc-802690486eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.626365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8c397-ff66-492d-bfdc-802690486eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:25.627057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8c397-ff66-492d-bfdc-802690486eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:25.629161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8c397-ff66-492d-bfdc-802690486eb3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:25.644974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 962/1937] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:25.651047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8c397-ff66-492d-bfdc-802690486eb3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:25.666098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8c397-ff66-492d-bfdc-802690486eb3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:25.675356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e35c1-375e-45c3-a0ee-05376716019f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:25.676108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e35c1-375e-45c3-a0ee-05376716019f admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.676653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e35c1-375e-45c3-a0ee-05376716019f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.676843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e35c1-375e-45c3-a0ee-05376716019f admin admin] PATH_INFO: `/v3/projects/443b0cf1604e4b74ab4f4db3ddb37bd8/users/076943a35907479f8759c423ed4871d9/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:25.679352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e35c1-375e-45c3-a0ee-05376716019f admin admin] RBAC: Authorizing `identity:create_grant(project_id=443b0cf1604e4b74ab4f4db3ddb37bd8, user_id=076943a35907479f8759c423ed4871d9, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:25.682739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e35c1-375e-45c3-a0ee-05376716019f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:25.733347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 963/1938] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:25 2022] PUT /identity/v3/projects/443b0cf1604e4b74ab4f4db3ddb37bd8/users/076943a35907479f8759c423ed4871d9/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:25.744396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aefc6d-be5e-4f5a-8c4e-5db07af95477 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:25.907063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:25.939301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c7b90-6261-4961-93bf-2b83c87185e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:25.939975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7b90-6261-4961-93bf-2b83c87185e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:25.940258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7b90-6261-4961-93bf-2b83c87185e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:25.940535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7b90-6261-4961-93bf-2b83c87185e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:26.116679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c7b90-6261-4961-93bf-2b83c87185e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:26.117498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e35c1-375e-45c3-a0ee-05376716019f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:26.118677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c7b90-6261-4961-93bf-2b83c87185e3 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:26.131341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 976/1939] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:31:26.295355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c7b90-6261-4961-93bf-2b83c87185e3 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:26.305720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c7b90-6261-4961-93bf-2b83c87185e3 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:26.318106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de692d4a-d945-4941-8697-316ea51c22fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:26.319869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de692d4a-d945-4941-8697-316ea51c22fd service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:26.319869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de692d4a-d945-4941-8697-316ea51c22fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:26.319869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de692d4a-d945-4941-8697-316ea51c22fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:26.338472 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e35c1-375e-45c3-a0ee-05376716019f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:26.368382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:26.369062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:26.369517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:26.369796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] PATH_INFO: `/v3/users/c53439fafa0a40ae8ae580e3916e3392` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:26.374138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c53439fafa0a40ae8ae580e3916e3392)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:26.376196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:26.447769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de692d4a-d945-4941-8697-316ea51c22fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:26.449916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de692d4a-d945-4941-8697-316ea51c22fd service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:26.474274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 977/1940] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:26.530209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c53439fafa0a40ae8ae580e3916e3392'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:26.531428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c53439fafa0a40ae8ae580e3916e3392'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:26.533054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c53439fafa0a40ae8ae580e3916e3392'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:26.550934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c53439fafa0a40ae8ae580e3916e3392'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:26.567582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c53439fafa0a40ae8ae580e3916e3392'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:26.571616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:26.601514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 964/1941] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:26 2022] DELETE /identity/v3/users/c53439fafa0a40ae8ae580e3916e3392 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:26.605689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccbc47-803b-4184-b0d0-39c0b324f7c9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:26.606439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccbc47-803b-4184-b0d0-39c0b324f7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:26.606630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccbc47-803b-4184-b0d0-39c0b324f7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:26.643963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-58ccbc47-803b-4184-b0d0-39c0b324f7c9 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:26.645279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-58ccbc47-803b-4184-b0d0-39c0b324f7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:26.791107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 978/1942] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:26.855617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:26.859240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:26.994633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b194d96-30b9-4917-a942-cd15be2a6241 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:27.008757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:27.011332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:27.011903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:27.012547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:27.030341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:27.032603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:27.051720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 965/1943] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:27.665671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b02fe7-3229-4f8a-9251-d7556ac39a82 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:27.667560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b02fe7-3229-4f8a-9251-d7556ac39a82 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:27.667980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b02fe7-3229-4f8a-9251-d7556ac39a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:27.728473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-14b02fe7-3229-4f8a-9251-d7556ac39a82 None None] MFA Rules not processed for user `076943a35907479f8759c423ed4871d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:27.731370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-14b02fe7-3229-4f8a-9251-d7556ac39a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:27.782996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:27.875499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:27.880041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:27.899769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ec39-de8c-4621-8cfa-4f37dec60a9c service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:27.910450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 979/1944] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:27.910717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:27.910717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:27.910717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:27.910717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] PATH_INFO: `/v3/projects/7f4aecbea25b4923b878f787b425f102` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:27.911329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4aecbea25b4923b878f787b425f102)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:27.911329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:27.919196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b02fe7-3229-4f8a-9251-d7556ac39a82 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:27.925883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-14b02fe7-3229-4f8a-9251-d7556ac39a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:27.981262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4aecbea25b4923b878f787b425f102', 'request_id': 'req-5b48447f-ace8-4869-a297-20ff780a40c7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:27.999845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4aecbea25b4923b878f787b425f102', 'request_id': 'req-5b48447f-ace8-4869-a297-20ff780a40c7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.008430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4aecbea25b4923b878f787b425f102', 'request_id': 'req-5b48447f-ace8-4869-a297-20ff780a40c7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.040127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] The token cache is being invalidate because project 7f4aecbea25b4923b878f787b425f102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:28.041085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.042589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.046425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 966/1945] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:27 2022] DELETE /identity/v3/projects/7f4aecbea25b4923b878f787b425f102 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:28.055197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:28.059741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:28.133532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b02fe7-3229-4f8a-9251-d7556ac39a82 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:28.284133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b48447f-ace8-4869-a297-20ff780a40c7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:28.307107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:28.308392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:28.308991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:28.311345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:28.317210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:28.322249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:28.323249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:28.323679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:28.324128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] PATH_INFO: `/v3/users/f8bb173a4ffa47b3b621749c214b4d7f` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:28.327393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8bb173a4ffa47b3b621749c214b4d7f)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:28.332356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:28.367773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:28.457839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8bb173a4ffa47b3b621749c214b4d7f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.457839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8bb173a4ffa47b3b621749c214b4d7f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.457839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8bb173a4ffa47b3b621749c214b4d7f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.469654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:28.471842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8bb173a4ffa47b3b621749c214b4d7f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.472296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:28.485822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 980/1946] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:28.517168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-215c45f1-5040-4274-a848-3ff717922dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8bb173a4ffa47b3b621749c214b4d7f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:28.561108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 967/1947] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:28 2022] DELETE /identity/v3/users/f8bb173a4ffa47b3b621749c214b4d7f => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:29.729554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:29.765240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500f615-2888-49ec-a713-c28cc169918d service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:29.846913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:29.975937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:29.976877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:29.977185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:29.985741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] PATH_INFO: `/v3/projects/ef1717ff2ccd4295b9d5194c692308af` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:29.991795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1717ff2ccd4295b9d5194c692308af)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:29.994330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:30.186202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1717ff2ccd4295b9d5194c692308af', 'request_id': 'req-44e2546b-f80e-4542-8921-981175c6887b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:30.213337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1717ff2ccd4295b9d5194c692308af', 'request_id': 'req-44e2546b-f80e-4542-8921-981175c6887b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:30.230549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1717ff2ccd4295b9d5194c692308af', 'request_id': 'req-44e2546b-f80e-4542-8921-981175c6887b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:30.266508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] The token cache is being invalidate because project ef1717ff2ccd4295b9d5194c692308af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:30.267119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:30.269619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-44e2546b-f80e-4542-8921-981175c6887b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:30.273896 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 981/1948] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:29 2022] DELETE /identity/v3/projects/ef1717ff2ccd4295b9d5194c692308af => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:30.280888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e919651-c152-428b-b621-ed20411d290a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:30.281514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e919651-c152-428b-b621-ed20411d290a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:30.281979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e919651-c152-428b-b621-ed20411d290a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:30.330008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2e919651-c152-428b-b621-ed20411d290a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:30.331069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2e919651-c152-428b-b621-ed20411d290a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:30.499176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 968/1949] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:30.509162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352bc97-b27f-46fe-9205-7975c27284dd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:30.509627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352bc97-b27f-46fe-9205-7975c27284dd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:30.510070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352bc97-b27f-46fe-9205-7975c27284dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:30.564143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0352bc97-b27f-46fe-9205-7975c27284dd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:30.566743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0352bc97-b27f-46fe-9205-7975c27284dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:30.588992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 982/1950] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:30.595663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e919651-c152-428b-b621-ed20411d290a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:30.610283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e919651-c152-428b-b621-ed20411d290a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:30.616810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:30.617425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:30.617811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:30.618075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:30.618737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:30.619041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:30.621399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:30.656074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 969/1951] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:30.663497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352bc97-b27f-46fe-9205-7975c27284dd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:30.687010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352bc97-b27f-46fe-9205-7975c27284dd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:30.694601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:30.697244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:30.699184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c31e94-6660-4626-a400-8b2cbe8a5942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:30.699851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c31e94-6660-4626-a400-8b2cbe8a5942 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:30.700140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c31e94-6660-4626-a400-8b2cbe8a5942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:30.700539 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c31e94-6660-4626-a400-8b2cbe8a5942 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:30.701351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c31e94-6660-4626-a400-8b2cbe8a5942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:30.707412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c31e94-6660-4626-a400-8b2cbe8a5942 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:30.826429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 983/1952] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:30 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:30.830988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c31e94-6660-4626-a400-8b2cbe8a5942 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:30.897813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef3ec8-0a77-45fe-b1f4-fc5f6edc216a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:31.027726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:31.044285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c31e94-6660-4626-a400-8b2cbe8a5942 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:31.055356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0202d-3c05-4c12-a480-4a282af51ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:31.056078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0202d-3c05-4c12-a480-4a282af51ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:31.056372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0202d-3c05-4c12-a480-4a282af51ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:31.056649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0202d-3c05-4c12-a480-4a282af51ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:31.057324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0202d-3c05-4c12-a480-4a282af51ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:31.059686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0202d-3c05-4c12-a480-4a282af51ea9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:31.073763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35c6eb-aaf8-4f39-b0b7-de68c855cb5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:31.074633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35c6eb-aaf8-4f39-b0b7-de68c855cb5f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:31.074805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35c6eb-aaf8-4f39-b0b7-de68c855cb5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:31.074972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35c6eb-aaf8-4f39-b0b7-de68c855cb5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:31.077582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cc35c6eb-aaf8-4f39-b0b7-de68c855cb5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:31.109021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 984/1953] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:30 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:31.115258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0202d-3c05-4c12-a480-4a282af51ea9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:31.142878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0202d-3c05-4c12-a480-4a282af51ea9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:31.174401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceb77b-e4c8-49b0-bc4a-ec65bbc912a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:31.175952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceb77b-e4c8-49b0-bc4a-ec65bbc912a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:31.176733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceb77b-e4c8-49b0-bc4a-ec65bbc912a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:31.177220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceb77b-e4c8-49b0-bc4a-ec65bbc912a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:31.178282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baceb77b-e4c8-49b0-bc4a-ec65bbc912a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:31.180788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baceb77b-e4c8-49b0-bc4a-ec65bbc912a6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:31.208584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 985/1954] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:31.220009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceb77b-e4c8-49b0-bc4a-ec65bbc912a6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:31.250897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceb77b-e4c8-49b0-bc4a-ec65bbc912a6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:31.262457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba724d-5581-4f38-b9c1-0278a932c8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:31.263638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba724d-5581-4f38-b9c1-0278a932c8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:31.263994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba724d-5581-4f38-b9c1-0278a932c8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:31.266626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba724d-5581-4f38-b9c1-0278a932c8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:31.266626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba724d-5581-4f38-b9c1-0278a932c8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:31.270083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba724d-5581-4f38-b9c1-0278a932c8d8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:31.302946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 986/1955] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:31.313922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba724d-5581-4f38-b9c1-0278a932c8d8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:31.323270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:31.355464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35c6eb-aaf8-4f39-b0b7-de68c855cb5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:31.361359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35c6eb-aaf8-4f39-b0b7-de68c855cb5f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:31.369406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba724d-5581-4f38-b9c1-0278a932c8d8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:31.407555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 970/1956] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:31.409682 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40b583-6732-4790-af5e-19746ea83800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:31.410488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40b583-6732-4790-af5e-19746ea83800 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:31.410682 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40b583-6732-4790-af5e-19746ea83800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:31.411006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40b583-6732-4790-af5e-19746ea83800 admin admin] PATH_INFO: `/v3/projects/cc992c50758d48c890ca42059912a4ae/users/a570d325ef3e45088f7880c8c20f2ec9/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:31.440001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40b583-6732-4790-af5e-19746ea83800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc992c50758d48c890ca42059912a4ae, user_id=a570d325ef3e45088f7880c8c20f2ec9, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:31.446815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40b583-6732-4790-af5e-19746ea83800 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:31.481496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 987/1957] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:31 2022] PUT /identity/v3/projects/cc992c50758d48c890ca42059912a4ae/users/a570d325ef3e45088f7880c8c20f2ec9/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:31.492204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8b942-7a71-45c7-b237-29f597eb780a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:31.492842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8b942-7a71-45c7-b237-29f597eb780a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:31.492974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8b942-7a71-45c7-b237-29f597eb780a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:31.574981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-51d8b942-7a71-45c7-b237-29f597eb780a None None] MFA Rules not processed for user `a570d325ef3e45088f7880c8c20f2ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:31.579038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-51d8b942-7a71-45c7-b237-29f597eb780a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:31.728791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40b583-6732-4790-af5e-19746ea83800 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:31.738376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 971/1958] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:31.931523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40b583-6732-4790-af5e-19746ea83800 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:31.950287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ab35b-3eca-4036-9d58-cef688f94fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:31.953277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab35b-3eca-4036-9d58-cef688f94fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:31.953746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab35b-3eca-4036-9d58-cef688f94fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:31.954002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ab35b-3eca-4036-9d58-cef688f94fe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:31.956518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ab35b-3eca-4036-9d58-cef688f94fe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:31.961822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ab35b-3eca-4036-9d58-cef688f94fe2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:32.033187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 988/1959] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:31 2022] POST /identity/v3/projects => generated 375 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:32.037732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8b942-7a71-45c7-b237-29f597eb780a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:32.066938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8b942-7a71-45c7-b237-29f597eb780a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:32.255143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbf93d-5d11-4bf6-9962-6396f827dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:32.255800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf93d-5d11-4bf6-9962-6396f827dcff admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:32.256091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf93d-5d11-4bf6-9962-6396f827dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:32.256429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbf93d-5d11-4bf6-9962-6396f827dcff admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:32.257105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbf93d-5d11-4bf6-9962-6396f827dcff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:32.259226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbf93d-5d11-4bf6-9962-6396f827dcff admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:32.288349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 972/1960] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:32 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:32.294152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ab35b-3eca-4036-9d58-cef688f94fe2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:32.314276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ab35b-3eca-4036-9d58-cef688f94fe2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:32.323238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d775cf-7de4-401d-8651-461057cb8f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:32.324301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d775cf-7de4-401d-8651-461057cb8f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:32.324834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d775cf-7de4-401d-8651-461057cb8f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:32.325339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d775cf-7de4-401d-8651-461057cb8f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:32.327886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d775cf-7de4-401d-8651-461057cb8f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:32.330777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d775cf-7de4-401d-8651-461057cb8f2b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:32.354534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 989/1961] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:31:32.359420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbf93d-5d11-4bf6-9962-6396f827dcff admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:32.370619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbf93d-5d11-4bf6-9962-6396f827dcff admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:32.375892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d378176-3bf1-4901-a1fa-26e063350d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:32.376745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378176-3bf1-4901-a1fa-26e063350d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:32.377161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378176-3bf1-4901-a1fa-26e063350d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:32.377600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378176-3bf1-4901-a1fa-26e063350d76 admin admin] PATH_INFO: `/v3/projects/93eb70513c654d53b6378957f609bca8/users/d27176299e2040aaacdd2ff7f963c03e/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:32.389149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:32.398877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d378176-3bf1-4901-a1fa-26e063350d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eb70513c654d53b6378957f609bca8, user_id=d27176299e2040aaacdd2ff7f963c03e, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:32.401827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d378176-3bf1-4901-a1fa-26e063350d76 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:32.422243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 973/1962] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:32 2022] PUT /identity/v3/projects/93eb70513c654d53b6378957f609bca8/users/d27176299e2040aaacdd2ff7f963c03e/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:32.427391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d775cf-7de4-401d-8651-461057cb8f2b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:32.441533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d775cf-7de4-401d-8651-461057cb8f2b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:32.517688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e40556-c36a-4f00-b0d2-5ee5d00ba950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:32.518353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e40556-c36a-4f00-b0d2-5ee5d00ba950 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:32.518639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e40556-c36a-4f00-b0d2-5ee5d00ba950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:32.518923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e40556-c36a-4f00-b0d2-5ee5d00ba950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:32.519614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e40556-c36a-4f00-b0d2-5ee5d00ba950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:32.521333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e40556-c36a-4f00-b0d2-5ee5d00ba950 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:32.536552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 990/1963] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:32.541842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d378176-3bf1-4901-a1fa-26e063350d76 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:32.557744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d378176-3bf1-4901-a1fa-26e063350d76 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:32.566533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:32.567705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:32.568218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:32.568740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] PATH_INFO: `/v3/projects/93eb70513c654d53b6378957f609bca8/users/d27176299e2040aaacdd2ff7f963c03e/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:32.572389 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=93eb70513c654d53b6378957f609bca8, user_id=d27176299e2040aaacdd2ff7f963c03e, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:32.578624 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:32.598295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 974/1964] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:32 2022] PUT /identity/v3/projects/93eb70513c654d53b6378957f609bca8/users/d27176299e2040aaacdd2ff7f963c03e/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:32.616197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcc454-b490-44b0-8960-2b9be3d85dca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:32.616546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcc454-b490-44b0-8960-2b9be3d85dca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:32.617127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcc454-b490-44b0-8960-2b9be3d85dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:32.653249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aadcc454-b490-44b0-8960-2b9be3d85dca None None] MFA Rules not processed for user `d27176299e2040aaacdd2ff7f963c03e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:32.655203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aadcc454-b490-44b0-8960-2b9be3d85dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:32.709629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:32.781023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 991/1965] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:32.789324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:32.791570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:32.870911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349933d-552c-47dd-a669-7ad5e84d5dea admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:32.876601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedcd68-66bd-4916-bc96-78658aa6bd70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:32.877679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedcd68-66bd-4916-bc96-78658aa6bd70 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:32.879151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedcd68-66bd-4916-bc96-78658aa6bd70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:32.879702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedcd68-66bd-4916-bc96-78658aa6bd70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:32.977242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedcd68-66bd-4916-bc96-78658aa6bd70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:32.980421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedcd68-66bd-4916-bc96-78658aa6bd70 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:32.999795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 975/1966] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:39.570843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd47b2-dcad-4654-abdb-6f1bf6672615 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:39.571450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd47b2-dcad-4654-abdb-6f1bf6672615 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:39.572044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd47b2-dcad-4654-abdb-6f1bf6672615 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:39.575035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 992/1967] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:31:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:31:39.672399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181192d2-be9c-417e-8925-6f9980cfe6fd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:39.673435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181192d2-be9c-417e-8925-6f9980cfe6fd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:39.673915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181192d2-be9c-417e-8925-6f9980cfe6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:39.714689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-181192d2-be9c-417e-8925-6f9980cfe6fd None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:39.718218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-181192d2-be9c-417e-8925-6f9980cfe6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:39.936221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 976/1968] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:40.039514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd47b2-dcad-4654-abdb-6f1bf6672615 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:40.041871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bdcd47b2-dcad-4654-abdb-6f1bf6672615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:40.103113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd47b2-dcad-4654-abdb-6f1bf6672615 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:40.128555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d71724-fc62-453a-8940-7afc63ae179e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:40.129330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d71724-fc62-453a-8940-7afc63ae179e service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:40.129663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d71724-fc62-453a-8940-7afc63ae179e service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:40.129951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d71724-fc62-453a-8940-7afc63ae179e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:40.190533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d71724-fc62-453a-8940-7afc63ae179e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:40.192951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d71724-fc62-453a-8940-7afc63ae179e service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:40.221952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 993/1969] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:41.439887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181192d2-be9c-417e-8925-6f9980cfe6fd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:41.482046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:41.589745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181192d2-be9c-417e-8925-6f9980cfe6fd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:41.597495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:41.598104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:41.598336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:41.598756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] PATH_INFO: `/v3/users/a570d325ef3e45088f7880c8c20f2ec9` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:41.601728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a570d325ef3e45088f7880c8c20f2ec9)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:41.604189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:41.695221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a570d325ef3e45088f7880c8c20f2ec9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:41.696432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a570d325ef3e45088f7880c8c20f2ec9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:41.697233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a570d325ef3e45088f7880c8c20f2ec9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:41.715533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a570d325ef3e45088f7880c8c20f2ec9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:41.762249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a570d325ef3e45088f7880c8c20f2ec9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:41.794612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 977/1970] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:41 2022] DELETE /identity/v3/users/a570d325ef3e45088f7880c8c20f2ec9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:41.798158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a0e9e-a29f-4ec2-b31f-0cd781446425 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:41.798757 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a0e9e-a29f-4ec2-b31f-0cd781446425 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:41.799168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a0e9e-a29f-4ec2-b31f-0cd781446425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:41.840520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0e3a0e9e-a29f-4ec2-b31f-0cd781446425 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:41.842685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0e3a0e9e-a29f-4ec2-b31f-0cd781446425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:41.972050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 994/1971] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:41.991572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:41.993900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:42.108580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22950a-a965-40eb-af86-eedf683d16d3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:42.115948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4a2f2-0b72-46b5-8a2b-197e7e4c9d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:42.117027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4a2f2-0b72-46b5-8a2b-197e7e4c9d65 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:42.117358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4a2f2-0b72-46b5-8a2b-197e7e4c9d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:42.117757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4a2f2-0b72-46b5-8a2b-197e7e4c9d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:42.131563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4a2f2-0b72-46b5-8a2b-197e7e4c9d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:42.133992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4a2f2-0b72-46b5-8a2b-197e7e4c9d65 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:42.147401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 978/1972] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:42.745791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a0e9e-a29f-4ec2-b31f-0cd781446425 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:42.752016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0e3a0e9e-a29f-4ec2-b31f-0cd781446425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:42.771714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a0e9e-a29f-4ec2-b31f-0cd781446425 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:42.776698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:42.777371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:42.779225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:42.779855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] PATH_INFO: `/v3/projects/cc992c50758d48c890ca42059912a4ae` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:42.782427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc992c50758d48c890ca42059912a4ae)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:42.789437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:42.880423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc992c50758d48c890ca42059912a4ae', 'request_id': 'req-c381ee72-80e7-4769-94e6-48b1db9659fd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:42.892139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc992c50758d48c890ca42059912a4ae', 'request_id': 'req-c381ee72-80e7-4769-94e6-48b1db9659fd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:42.905413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc992c50758d48c890ca42059912a4ae', 'request_id': 'req-c381ee72-80e7-4769-94e6-48b1db9659fd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:42.933181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] The token cache is being invalidate because project cc992c50758d48c890ca42059912a4ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:42.933789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:42.936222 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:42.938781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 995/1973] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:42 2022] DELETE /identity/v3/projects/cc992c50758d48c890ca42059912a4ae => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:42.943872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4a2f2-0b72-46b5-8a2b-197e7e4c9d65 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:42.961827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4a2f2-0b72-46b5-8a2b-197e7e4c9d65 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:43.032486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:43.083641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:43.085168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:43.085722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:43.086224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] PATH_INFO: `/v3/users/d27176299e2040aaacdd2ff7f963c03e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:43.088479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27176299e2040aaacdd2ff7f963c03e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:43.093718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:43.211264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27176299e2040aaacdd2ff7f963c03e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:43.212530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27176299e2040aaacdd2ff7f963c03e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:43.213938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27176299e2040aaacdd2ff7f963c03e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:43.230287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27176299e2040aaacdd2ff7f963c03e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:43.258936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-74e3ee58-42a3-46f9-bdfc-a18db31d231d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27176299e2040aaacdd2ff7f963c03e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:43.285840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 979/1974] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:42 2022] DELETE /identity/v3/users/d27176299e2040aaacdd2ff7f963c03e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:43.695805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:43.697622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:43.715570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c381ee72-80e7-4769-94e6-48b1db9659fd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:43.807477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:43.867268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:43.869479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:43.869995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:43.870652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] PATH_INFO: `/v3/projects/93eb70513c654d53b6378957f609bca8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:43.880900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=93eb70513c654d53b6378957f609bca8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:43.885473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:44.101792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93eb70513c654d53b6378957f609bca8', 'request_id': 'req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:44.134585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93eb70513c654d53b6378957f609bca8', 'request_id': 'req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:44.168716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93eb70513c654d53b6378957f609bca8', 'request_id': 'req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:44.246013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] The token cache is being invalidate because project 93eb70513c654d53b6378957f609bca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:44.247985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:44.250342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c0c8af6b-1b69-44d8-a1cc-937458fce6cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:44.254298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 996/1975] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:43 2022] DELETE /identity/v3/projects/93eb70513c654d53b6378957f609bca8 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:44.262470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746409-bf7f-4694-8228-9a67d0fa8b3f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:44.264972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746409-bf7f-4694-8228-9a67d0fa8b3f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:44.265761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f746409-bf7f-4694-8228-9a67d0fa8b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:44.349568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9f746409-bf7f-4694-8228-9a67d0fa8b3f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:44.351602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9f746409-bf7f-4694-8228-9a67d0fa8b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:44.453983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 980/1976] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:44.470334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2c7d-86d7-4d33-93a8-05458a157220 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:44.474935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2c7d-86d7-4d33-93a8-05458a157220 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:44.475688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2c7d-86d7-4d33-93a8-05458a157220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:44.533378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-93fa2c7d-86d7-4d33-93a8-05458a157220 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:44.535449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-93fa2c7d-86d7-4d33-93a8-05458a157220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:44.549709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 997/1977] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:44.556031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f746409-bf7f-4694-8228-9a67d0fa8b3f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:44.586954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f746409-bf7f-4694-8228-9a67d0fa8b3f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:44.597524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:44.597524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:44.597524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:44.598094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:44.599409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:44.600013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:44.602342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:44.624692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 981/1978] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:31:44.629883 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2c7d-86d7-4d33-93a8-05458a157220 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:44.645175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2c7d-86d7-4d33-93a8-05458a157220 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:44.650709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b809a-33e3-45da-9b4c-101250f1c8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:44.651514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b809a-33e3-45da-9b4c-101250f1c8eb admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:44.651900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b809a-33e3-45da-9b4c-101250f1c8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:44.652280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b809a-33e3-45da-9b4c-101250f1c8eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:44.653070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b809a-33e3-45da-9b4c-101250f1c8eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:44.656373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b809a-33e3-45da-9b4c-101250f1c8eb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:44.703420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 998/1979] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:44 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:44.710104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:44.727393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f36db7-422c-48ce-84e1-65c939855fa8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:44.804619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:44.838009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37520df-12f0-44fe-a57a-5179ecaa3caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:44.839636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37520df-12f0-44fe-a57a-5179ecaa3caa admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:44.840228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37520df-12f0-44fe-a57a-5179ecaa3caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:44.841379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37520df-12f0-44fe-a57a-5179ecaa3caa admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:44.842539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37520df-12f0-44fe-a57a-5179ecaa3caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:44.845787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37520df-12f0-44fe-a57a-5179ecaa3caa admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:44.885641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 982/1980] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:44 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:44.891340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b809a-33e3-45da-9b4c-101250f1c8eb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:44.913104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b809a-33e3-45da-9b4c-101250f1c8eb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:44.918150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6258fb-507e-4dc3-892a-a399388cc87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:44.919573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6258fb-507e-4dc3-892a-a399388cc87a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:44.920112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6258fb-507e-4dc3-892a-a399388cc87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:44.920594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6258fb-507e-4dc3-892a-a399388cc87a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:44.921843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6258fb-507e-4dc3-892a-a399388cc87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:44.924355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6258fb-507e-4dc3-892a-a399388cc87a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:44.938709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 999/1981] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:44.943359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37520df-12f0-44fe-a57a-5179ecaa3caa admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:44.955954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37520df-12f0-44fe-a57a-5179ecaa3caa admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:44.961559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165493d-34e7-4396-a956-d51b41dcd351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:44.962197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165493d-34e7-4396-a956-d51b41dcd351 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:44.962480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165493d-34e7-4396-a956-d51b41dcd351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:44.962774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165493d-34e7-4396-a956-d51b41dcd351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:44.963498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165493d-34e7-4396-a956-d51b41dcd351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:44.965268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165493d-34e7-4396-a956-d51b41dcd351 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:44.985744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 983/1982] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:31:44.990323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6258fb-507e-4dc3-892a-a399388cc87a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:45.006723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6258fb-507e-4dc3-892a-a399388cc87a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:45.011895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75f72e-60fb-4ba7-946b-9dc57d90d6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:45.013479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75f72e-60fb-4ba7-946b-9dc57d90d6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:45.013479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75f72e-60fb-4ba7-946b-9dc57d90d6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:45.014045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75f72e-60fb-4ba7-946b-9dc57d90d6cc admin admin] PATH_INFO: `/v3/projects/9935db9afd624fbb9c5a1da9273d60ed/users/900de1d2272d4ffb8c0d60163ee5e5b8/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:45.043075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75f72e-60fb-4ba7-946b-9dc57d90d6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9935db9afd624fbb9c5a1da9273d60ed, user_id=900de1d2272d4ffb8c0d60163ee5e5b8, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:45.046178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75f72e-60fb-4ba7-946b-9dc57d90d6cc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:45.069566 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1000/1983] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:44 2022] PUT /identity/v3/projects/9935db9afd624fbb9c5a1da9273d60ed/users/900de1d2272d4ffb8c0d60163ee5e5b8/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:45.078395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313488b-cfa8-4d34-abfb-5941dbf5b156 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:45.078934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313488b-cfa8-4d34-abfb-5941dbf5b156 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:45.079420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313488b-cfa8-4d34-abfb-5941dbf5b156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:45.120520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1313488b-cfa8-4d34-abfb-5941dbf5b156 None None] MFA Rules not processed for user `900de1d2272d4ffb8c0d60163ee5e5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:45.123078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1313488b-cfa8-4d34-abfb-5941dbf5b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:45.257775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 984/1984] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:45.261331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75f72e-60fb-4ba7-946b-9dc57d90d6cc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:45.433215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75f72e-60fb-4ba7-946b-9dc57d90d6cc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:45.446030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb50a82-74dd-4b7f-a77c-1bc22d2fe651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:45.449308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb50a82-74dd-4b7f-a77c-1bc22d2fe651 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:45.451618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb50a82-74dd-4b7f-a77c-1bc22d2fe651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:45.462224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb50a82-74dd-4b7f-a77c-1bc22d2fe651 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:45.463243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb50a82-74dd-4b7f-a77c-1bc22d2fe651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:45.466488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb50a82-74dd-4b7f-a77c-1bc22d2fe651 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:45.521383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:45.591038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1001/1985] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:45 2022] POST /identity/v3/projects => generated 385 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:45.598710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313488b-cfa8-4d34-abfb-5941dbf5b156 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:45.634101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313488b-cfa8-4d34-abfb-5941dbf5b156 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:45.830444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e353d2-c72a-4f6d-9e1c-276a7b4ff012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:45.831256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e353d2-c72a-4f6d-9e1c-276a7b4ff012 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:45.832148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e353d2-c72a-4f6d-9e1c-276a7b4ff012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:45.832439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e353d2-c72a-4f6d-9e1c-276a7b4ff012 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:45.833107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e353d2-c72a-4f6d-9e1c-276a7b4ff012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:45.835119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e353d2-c72a-4f6d-9e1c-276a7b4ff012 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:45.877010 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 985/1986] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:45 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:45.881677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb50a82-74dd-4b7f-a77c-1bc22d2fe651 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:45.915225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb50a82-74dd-4b7f-a77c-1bc22d2fe651 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:45.919796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775d51e-4fb3-41e3-a429-5667613e5240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:45.921216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775d51e-4fb3-41e3-a429-5667613e5240 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:45.922232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775d51e-4fb3-41e3-a429-5667613e5240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:45.922232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775d51e-4fb3-41e3-a429-5667613e5240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:45.923146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775d51e-4fb3-41e3-a429-5667613e5240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:45.925710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775d51e-4fb3-41e3-a429-5667613e5240 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:45.942774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1002/1987] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:31:45.948616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e353d2-c72a-4f6d-9e1c-276a7b4ff012 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:45.960681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e353d2-c72a-4f6d-9e1c-276a7b4ff012 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:45.967103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:45.967903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:45.968343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:45.968597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:45.969327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:45.971179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:45.994249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 986/1988] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:46.000666 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775d51e-4fb3-41e3-a429-5667613e5240 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:46.021202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775d51e-4fb3-41e3-a429-5667613e5240 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:46.030144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693c7f6-8359-4be6-b6a7-775616ba198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:46.031273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c7f6-8359-4be6-b6a7-775616ba198f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.031793 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c7f6-8359-4be6-b6a7-775616ba198f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.032329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c7f6-8359-4be6-b6a7-775616ba198f admin admin] PATH_INFO: `/v3/projects/117dc432704c4c81b7c91c8c4b57e785/users/57d383978b7248298cc7e962b7b04615/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.096844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:46.099669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:46.106804 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693c7f6-8359-4be6-b6a7-775616ba198f admin admin] RBAC: Authorizing `identity:create_grant(project_id=117dc432704c4c81b7c91c8c4b57e785, user_id=57d383978b7248298cc7e962b7b04615, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:46.110322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693c7f6-8359-4be6-b6a7-775616ba198f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:46.117873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcfb9e-833d-456a-aada-cddc92101ae3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:46.125496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:46.126374 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.126665 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.126945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] PATH_INFO: `/v3/users/5a1e98f27b724b2a993a4ee96e9e024e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.129790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a1e98f27b724b2a993a4ee96e9e024e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:46.131792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1003/1989] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:45 2022] PUT /identity/v3/projects/117dc432704c4c81b7c91c8c4b57e785/users/57d383978b7248298cc7e962b7b04615/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:46.144144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a708a3d-374f-41ea-836a-6ec0ea652ba6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.144455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a708a3d-374f-41ea-836a-6ec0ea652ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.144889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a708a3d-374f-41ea-836a-6ec0ea652ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.146784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:46.192821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6a708a3d-374f-41ea-836a-6ec0ea652ba6 None None] MFA Rules not processed for user `57d383978b7248298cc7e962b7b04615`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:46.195875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6a708a3d-374f-41ea-836a-6ec0ea652ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:46.219300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:46.247989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a1e98f27b724b2a993a4ee96e9e024e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:46.249588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a1e98f27b724b2a993a4ee96e9e024e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:46.250759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a1e98f27b724b2a993a4ee96e9e024e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:46.264978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a1e98f27b724b2a993a4ee96e9e024e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:46.284444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-070d1dbf-1a61-43cd-b864-e2557d30a73c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a1e98f27b724b2a993a4ee96e9e024e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:46.305920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 987/1990] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:46 2022] DELETE /identity/v3/users/5a1e98f27b724b2a993a4ee96e9e024e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:46.314155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f8bb4-9bf2-4f09-97da-9157bfe8cccd None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.315083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f8bb4-9bf2-4f09-97da-9157bfe8cccd None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.315533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f8bb4-9bf2-4f09-97da-9157bfe8cccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.349803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c88f8bb4-9bf2-4f09-97da-9157bfe8cccd None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:46.351571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c88f8bb4-9bf2-4f09-97da-9157bfe8cccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:46.356521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1004/1991] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:46.362588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a708a3d-374f-41ea-836a-6ec0ea652ba6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:46.365380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6a708a3d-374f-41ea-836a-6ec0ea652ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:46.459081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 988/1992] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:46.473394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f8bb4-9bf2-4f09-97da-9157bfe8cccd None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:46.475043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c88f8bb4-9bf2-4f09-97da-9157bfe8cccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:46.501716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a708a3d-374f-41ea-836a-6ec0ea652ba6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:46.507863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41474b-0e3e-4d8d-9e08-da6869ac807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:46.508644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41474b-0e3e-4d8d-9e08-da6869ac807d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.508984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41474b-0e3e-4d8d-9e08-da6869ac807d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.509339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41474b-0e3e-4d8d-9e08-da6869ac807d admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.510115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41474b-0e3e-4d8d-9e08-da6869ac807d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:46.512284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41474b-0e3e-4d8d-9e08-da6869ac807d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:46.553556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1005/1993] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:46 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:46.556424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41474b-0e3e-4d8d-9e08-da6869ac807d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:46.560782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f8bb4-9bf2-4f09-97da-9157bfe8cccd None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:46.621751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:46.656112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd3fa8-8fea-439e-a6c6-1ad445d5d274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:46.657122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd3fa8-8fea-439e-a6c6-1ad445d5d274 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.657658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd3fa8-8fea-439e-a6c6-1ad445d5d274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.658016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd3fa8-8fea-439e-a6c6-1ad445d5d274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.668580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41474b-0e3e-4d8d-9e08-da6869ac807d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:46.671245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd3fa8-8fea-439e-a6c6-1ad445d5d274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:46.672828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd3fa8-8fea-439e-a6c6-1ad445d5d274 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:46.686278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 989/1994] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:46.686939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239164e0-4fd1-4080-8c21-04839918ff57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:46.687997 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239164e0-4fd1-4080-8c21-04839918ff57 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.688700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239164e0-4fd1-4080-8c21-04839918ff57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.689204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239164e0-4fd1-4080-8c21-04839918ff57 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.690143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239164e0-4fd1-4080-8c21-04839918ff57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:46.692996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239164e0-4fd1-4080-8c21-04839918ff57 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:46.725598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1006/1995] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:46 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:46.730565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd3fa8-8fea-439e-a6c6-1ad445d5d274 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:46.750939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd3fa8-8fea-439e-a6c6-1ad445d5d274 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:46.760854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134fd16-52c9-4e5d-bdf2-28fd40dcec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:46.761962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134fd16-52c9-4e5d-bdf2-28fd40dcec31 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.762992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134fd16-52c9-4e5d-bdf2-28fd40dcec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.763766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134fd16-52c9-4e5d-bdf2-28fd40dcec31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.765039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134fd16-52c9-4e5d-bdf2-28fd40dcec31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:46.768216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134fd16-52c9-4e5d-bdf2-28fd40dcec31 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:46.770913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:46.795481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239164e0-4fd1-4080-8c21-04839918ff57 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:46.804127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 990/1996] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:46.832889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239164e0-4fd1-4080-8c21-04839918ff57 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:46.850987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610093-d6cf-41ce-9a75-60ad8310cfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:46.851705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610093-d6cf-41ce-9a75-60ad8310cfe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:46.854136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610093-d6cf-41ce-9a75-60ad8310cfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:46.854463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610093-d6cf-41ce-9a75-60ad8310cfe7 admin admin] PATH_INFO: `/v3/projects/7c5d5a21bd8d4596bde9f0b6dd426e7f/users/fbb62dc3060a4e479bb2112cc946120b/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:46.890140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19610093-d6cf-41ce-9a75-60ad8310cfe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5d5a21bd8d4596bde9f0b6dd426e7f, user_id=fbb62dc3060a4e479bb2112cc946120b, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:46.893085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19610093-d6cf-41ce-9a75-60ad8310cfe7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:46.915909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1007/1997] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:46 2022] PUT /identity/v3/projects/7c5d5a21bd8d4596bde9f0b6dd426e7f/users/fbb62dc3060a4e479bb2112cc946120b/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:46.921770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134fd16-52c9-4e5d-bdf2-28fd40dcec31 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:46.939106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134fd16-52c9-4e5d-bdf2-28fd40dcec31 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:47.138772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2ccef-f6af-4fa2-8fc1-eb32414381d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:47.139420 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2ccef-f6af-4fa2-8fc1-eb32414381d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:47.139704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2ccef-f6af-4fa2-8fc1-eb32414381d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:47.139981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2ccef-f6af-4fa2-8fc1-eb32414381d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:47.140646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2ccef-f6af-4fa2-8fc1-eb32414381d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:47.142346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2ccef-f6af-4fa2-8fc1-eb32414381d5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:47.159521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610093-d6cf-41ce-9a75-60ad8310cfe7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:47.160523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 991/1998] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:47.177117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610093-d6cf-41ce-9a75-60ad8310cfe7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:47.182440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:47.183146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:47.183617 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:47.184053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] PATH_INFO: `/v3/projects/7c5d5a21bd8d4596bde9f0b6dd426e7f/users/fbb62dc3060a4e479bb2112cc946120b/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:47.186934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c5d5a21bd8d4596bde9f0b6dd426e7f, user_id=fbb62dc3060a4e479bb2112cc946120b, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:47.190182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:47.204479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1008/1999] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:47 2022] PUT /identity/v3/projects/7c5d5a21bd8d4596bde9f0b6dd426e7f/users/fbb62dc3060a4e479bb2112cc946120b/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:47.222266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e5ca7-f820-414e-b370-942632749d20 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:47.222835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e5ca7-f820-414e-b370-942632749d20 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:47.223275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e5ca7-f820-414e-b370-942632749d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:47.273741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-218e5ca7-f820-414e-b370-942632749d20 None None] MFA Rules not processed for user `fbb62dc3060a4e479bb2112cc946120b`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:47.275015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-218e5ca7-f820-414e-b370-942632749d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:47.445242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 992/2000] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:47.535198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:47.537302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:47.636501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e5ca7-f820-414e-b370-942632749d20 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:47.640522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-218e5ca7-f820-414e-b370-942632749d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:47.737372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099b129-1896-4cc2-a23e-83e5ba2cf2f7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:47.744594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:47.745656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e5ca7-f820-414e-b370-942632749d20 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:47.746595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:47.748887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:47.749407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] PATH_INFO: `/v3/projects/e2d99547a70949368d202ab42771e4e4` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:47.750900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:47.751339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d99547a70949368d202ab42771e4e4)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:47.751953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:47.752605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:47.753335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:47.756142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:47.756739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:47.824421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:47.863090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d99547a70949368d202ab42771e4e4', 'request_id': 'req-0a9dbed6-778d-472c-8cd0-775c6f99de76'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:47.869427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:47.871011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:47.879096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d99547a70949368d202ab42771e4e4', 'request_id': 'req-0a9dbed6-778d-472c-8cd0-775c6f99de76'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:47.904061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d99547a70949368d202ab42771e4e4', 'request_id': 'req-0a9dbed6-778d-472c-8cd0-775c6f99de76'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:47.910267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 993/2001] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:47.941075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] The token cache is being invalidate because project e2d99547a70949368d202ab42771e4e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:47.941881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:47.952929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:47.958070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1009/2002] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:47 2022] DELETE /identity/v3/projects/e2d99547a70949368d202ab42771e4e4 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:47.962495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:47.994111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66653bfc-9079-425b-b676-da6d55240bbe service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:48.188128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:48.188968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:48.189253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:48.189638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] PATH_INFO: `/v3/users/e338101ac6944cd4b64f2eae7e9453fb` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:48.191070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e338101ac6944cd4b64f2eae7e9453fb)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:48.193202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:48.310904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e338101ac6944cd4b64f2eae7e9453fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:48.312230 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e338101ac6944cd4b64f2eae7e9453fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:48.313322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e338101ac6944cd4b64f2eae7e9453fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:48.331883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e338101ac6944cd4b64f2eae7e9453fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:48.369103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e338101ac6944cd4b64f2eae7e9453fb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:48.413792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 994/2003] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:47 2022] DELETE /identity/v3/users/e338101ac6944cd4b64f2eae7e9453fb => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:48.661521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:48.664775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:48.713800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dbed6-778d-472c-8cd0-775c6f99de76 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:48.785551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:48.876362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:48.877202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:48.877654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:48.878305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] PATH_INFO: `/v3/users/3e6f478548c94ceea676aa4ab8ad6356` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:48.879792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6f478548c94ceea676aa4ab8ad6356)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:48.881967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:48.980469 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6f478548c94ceea676aa4ab8ad6356'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.009591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.011603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:49.018457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6f478548c94ceea676aa4ab8ad6356'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.019382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6f478548c94ceea676aa4ab8ad6356'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.040591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222ede3-40d5-44b7-b3c7-0d98d0b8f473 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.056082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6f478548c94ceea676aa4ab8ad6356'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.062046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.064774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.065439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.066024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] PATH_INFO: `/v3/projects/2363bf836b9944fca276c28fb87f088c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.069660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2363bf836b9944fca276c28fb87f088c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.073899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:49.077573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-13f8b68c-0fbb-4af6-a3b7-cf484843a144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6f478548c94ceea676aa4ab8ad6356'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.090939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1010/2004] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:48 2022] DELETE /identity/v3/users/3e6f478548c94ceea676aa4ab8ad6356 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:49.098255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2057a-a5ce-4d42-83a4-6bda20f07207 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.098554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2057a-a5ce-4d42-83a4-6bda20f07207 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.099358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf2057a-a5ce-4d42-83a4-6bda20f07207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.129179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dcf2057a-a5ce-4d42-83a4-6bda20f07207 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:49.130987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dcf2057a-a5ce-4d42-83a4-6bda20f07207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:49.195985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:49.200676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2363bf836b9944fca276c28fb87f088c', 'request_id': 'req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.219104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2363bf836b9944fca276c28fb87f088c', 'request_id': 'req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.245254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2363bf836b9944fca276c28fb87f088c', 'request_id': 'req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.251093 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1011/2005] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:49.265353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2057a-a5ce-4d42-83a4-6bda20f07207 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.266964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dcf2057a-a5ce-4d42-83a4-6bda20f07207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:49.270743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] The token cache is being invalidate because project 2363bf836b9944fca276c28fb87f088c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:49.272001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.274603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8d2fa0ef-5dac-48ce-bb00-9cf41927b05f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:49.282883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 995/2006] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:49 2022] DELETE /identity/v3/projects/2363bf836b9944fca276c28fb87f088c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:49.285380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8d823-8ed5-45e6-8e5b-0d2609efdef9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.285785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8d823-8ed5-45e6-8e5b-0d2609efdef9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.286303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8d823-8ed5-45e6-8e5b-0d2609efdef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.327382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-03b8d823-8ed5-45e6-8e5b-0d2609efdef9 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:49.329433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-03b8d823-8ed5-45e6-8e5b-0d2609efdef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:49.343133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:49.375329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf2057a-a5ce-4d42-83a4-6bda20f07207 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.445351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.446401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.446877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.447345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.453528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 996/2007] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:49.453805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:49.460233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e1041-c285-4bfb-9c00-38b0ab09caad None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.460530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e1041-c285-4bfb-9c00-38b0ab09caad None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.460811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e1041-c285-4bfb-9c00-38b0ab09caad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.475112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.478068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:49.484629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5d5e1041-c285-4bfb-9c00-38b0ab09caad None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:49.485990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5d5e1041-c285-4bfb-9c00-38b0ab09caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:49.494862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 997/2008] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:49.496012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1012/2009] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:49.501375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e1041-c285-4bfb-9c00-38b0ab09caad None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.515097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e1041-c285-4bfb-9c00-38b0ab09caad None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.523763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.524614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.525137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.525582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.526370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:49.526867 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.530682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:49.569720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 998/2010] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:31:49.574112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.590002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fccdca-b025-49a3-92ba-a4818c63afe6 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.597705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af8f9b-270c-443a-9fe9-36bb62da7158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.598385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af8f9b-270c-443a-9fe9-36bb62da7158 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.598674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af8f9b-270c-443a-9fe9-36bb62da7158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.598952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af8f9b-270c-443a-9fe9-36bb62da7158 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.599633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af8f9b-270c-443a-9fe9-36bb62da7158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.601711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af8f9b-270c-443a-9fe9-36bb62da7158 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:49.643754 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1013/2011] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:49 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:49.645908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.666042 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4e92a-e6d0-4279-aa44-f0eefe2663c6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.771326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670de070-78c1-4bd7-ac00-30dbc9889329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.772025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670de070-78c1-4bd7-ac00-30dbc9889329 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.772336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670de070-78c1-4bd7-ac00-30dbc9889329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.772634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670de070-78c1-4bd7-ac00-30dbc9889329 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.773284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670de070-78c1-4bd7-ac00-30dbc9889329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.775335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670de070-78c1-4bd7-ac00-30dbc9889329 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:49.808813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af8f9b-270c-443a-9fe9-36bb62da7158 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.820064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 999/2012] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:49 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:49.823479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af8f9b-270c-443a-9fe9-36bb62da7158 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.829655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098db4e8-91ab-400e-8618-fcc8de7f303b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.830400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098db4e8-91ab-400e-8618-fcc8de7f303b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.830691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098db4e8-91ab-400e-8618-fcc8de7f303b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.831021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098db4e8-91ab-400e-8618-fcc8de7f303b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.831663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098db4e8-91ab-400e-8618-fcc8de7f303b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.834477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098db4e8-91ab-400e-8618-fcc8de7f303b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:49.851875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1014/2013] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:49.855605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670de070-78c1-4bd7-ac00-30dbc9889329 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.867095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670de070-78c1-4bd7-ac00-30dbc9889329 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.873634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.874279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.874565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.874845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.875586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.878077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:49.901411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1000/2014] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:49.907142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098db4e8-91ab-400e-8618-fcc8de7f303b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.928756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:49.928996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098db4e8-91ab-400e-8618-fcc8de7f303b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.931369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:49.937749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774287c-28ca-4f8e-a485-ad25578f0b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.939087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774287c-28ca-4f8e-a485-ad25578f0b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.939599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774287c-28ca-4f8e-a485-ad25578f0b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.940110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774287c-28ca-4f8e-a485-ad25578f0b1a admin admin] PATH_INFO: `/v3/projects/22fa6c08ed81470b86bdbab4c38c8fcb/users/05037938f5844812a55be6e77da8ab60/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.958097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7cbcc-72fb-4b81-b565-d21ed8b3a6a5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:49.968081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:49.969333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:49.969333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:49.970013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] PATH_INFO: `/v3/projects/37a67d8c857f4b58a227cc39dcdea404` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:49.974119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a67d8c857f4b58a227cc39dcdea404)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.976830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:49.979234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774287c-28ca-4f8e-a485-ad25578f0b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=22fa6c08ed81470b86bdbab4c38c8fcb, user_id=05037938f5844812a55be6e77da8ab60, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:49.998600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774287c-28ca-4f8e-a485-ad25578f0b1a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:50.025160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1015/2015] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:49 2022] PUT /identity/v3/projects/22fa6c08ed81470b86bdbab4c38c8fcb/users/05037938f5844812a55be6e77da8ab60/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:50.039908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869b482-33c1-44ed-aa3e-297a339b5e70 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:50.041715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869b482-33c1-44ed-aa3e-297a339b5e70 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:50.042508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869b482-33c1-44ed-aa3e-297a339b5e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:50.043143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:50.073216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a67d8c857f4b58a227cc39dcdea404', 'request_id': 'req-e539d8e6-c7b6-49ef-824e-b24cace66ff8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.084505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a67d8c857f4b58a227cc39dcdea404', 'request_id': 'req-e539d8e6-c7b6-49ef-824e-b24cace66ff8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.089284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1869b482-33c1-44ed-aa3e-297a339b5e70 None None] MFA Rules not processed for user `05037938f5844812a55be6e77da8ab60`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:50.091127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1869b482-33c1-44ed-aa3e-297a339b5e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:50.102955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a67d8c857f4b58a227cc39dcdea404', 'request_id': 'req-e539d8e6-c7b6-49ef-824e-b24cace66ff8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.125959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] The token cache is being invalidate because project 37a67d8c857f4b58a227cc39dcdea404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:50.126407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.128105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.131411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1001/2016] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:49 2022] DELETE /identity/v3/projects/37a67d8c857f4b58a227cc39dcdea404 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:50.137244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:50.140269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:50.181400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1016/2017] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:50.187338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869b482-33c1-44ed-aa3e-297a339b5e70 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:50.190316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1869b482-33c1-44ed-aa3e-297a339b5e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:50.249612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:50.296445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539d8e6-c7b6-49ef-824e-b24cace66ff8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:50.307824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:50.308170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869b482-33c1-44ed-aa3e-297a339b5e70 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:50.309947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:50.310421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:50.310924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] PATH_INFO: `/v3/users/6462f65a0cfc41c782669fd6c4804e3e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:50.313184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6462f65a0cfc41c782669fd6c4804e3e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:50.316286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984ead5-d2ab-4b49-ad9c-c4901d2b8d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:50.316560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984ead5-d2ab-4b49-ad9c-c4901d2b8d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:50.316849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984ead5-d2ab-4b49-ad9c-c4901d2b8d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:50.317129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9984ead5-d2ab-4b49-ad9c-c4901d2b8d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:50.317293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:50.317849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984ead5-d2ab-4b49-ad9c-c4901d2b8d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:50.320205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9984ead5-d2ab-4b49-ad9c-c4901d2b8d22 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:50.386535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1017/2018] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:50 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:50.388262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984ead5-d2ab-4b49-ad9c-c4901d2b8d22 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:50.408164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6462f65a0cfc41c782669fd6c4804e3e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.409035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6462f65a0cfc41c782669fd6c4804e3e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.409801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6462f65a0cfc41c782669fd6c4804e3e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.418380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6462f65a0cfc41c782669fd6c4804e3e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.436579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6462f65a0cfc41c782669fd6c4804e3e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:50.461123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1002/2019] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:50 2022] DELETE /identity/v3/users/6462f65a0cfc41c782669fd6c4804e3e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:50.474898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9984ead5-d2ab-4b49-ad9c-c4901d2b8d22 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:50.557786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ae8a-6cc5-429f-9b50-9b9d29e385d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:50.558501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ae8a-6cc5-429f-9b50-9b9d29e385d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:50.560425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ae8a-6cc5-429f-9b50-9b9d29e385d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:50.560425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43ae8a-6cc5-429f-9b50-9b9d29e385d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:50.561424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43ae8a-6cc5-429f-9b50-9b9d29e385d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:50.564012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43ae8a-6cc5-429f-9b50-9b9d29e385d6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:50.601550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1018/2020] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:50 2022] POST /identity/v3/users => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:50.603427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:50.605768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:50.634926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f050c1-e289-4e7a-92ae-bd63d605f124 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:50.644414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e375578-a047-4c68-b00b-41637c5833f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:50.645364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e375578-a047-4c68-b00b-41637c5833f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:50.646024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e375578-a047-4c68-b00b-41637c5833f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:50.646975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e375578-a047-4c68-b00b-41637c5833f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:50.648524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e375578-a047-4c68-b00b-41637c5833f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:50.652844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e375578-a047-4c68-b00b-41637c5833f3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:50.681598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1003/2021] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:50.697663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ae8a-6cc5-429f-9b50-9b9d29e385d6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:50.721813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43ae8a-6cc5-429f-9b50-9b9d29e385d6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:50.730295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab1af0-6985-4abb-95fc-7474548e8188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:50.732105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1af0-6985-4abb-95fc-7474548e8188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:50.732582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1af0-6985-4abb-95fc-7474548e8188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:50.733114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab1af0-6985-4abb-95fc-7474548e8188 admin admin] PATH_INFO: `/v3/projects/242d586d63f3499186a3116e161b6a0e/users/644fa17c5d284d3a86f54840efbc6408/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:50.760022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab1af0-6985-4abb-95fc-7474548e8188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=242d586d63f3499186a3116e161b6a0e, user_id=644fa17c5d284d3a86f54840efbc6408, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:50.765751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab1af0-6985-4abb-95fc-7474548e8188 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:50.805954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1019/2022] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:50 2022] PUT /identity/v3/projects/242d586d63f3499186a3116e161b6a0e/users/644fa17c5d284d3a86f54840efbc6408/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:50.810547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e375578-a047-4c68-b00b-41637c5833f3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:50.821631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e375578-a047-4c68-b00b-41637c5833f3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:50.849988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:50.921106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:50.921106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:50.921106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:50.921106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:50.921502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:50.923003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:50.944875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1004/2023] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:50.952359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab1af0-6985-4abb-95fc-7474548e8188 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:50.966030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab1af0-6985-4abb-95fc-7474548e8188 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:50.972777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74670080-4a08-4c34-8105-a23249c3b65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:50.973519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74670080-4a08-4c34-8105-a23249c3b65b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:50.973876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74670080-4a08-4c34-8105-a23249c3b65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:50.974290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74670080-4a08-4c34-8105-a23249c3b65b admin admin] PATH_INFO: `/v3/projects/242d586d63f3499186a3116e161b6a0e/users/644fa17c5d284d3a86f54840efbc6408/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:50.976568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74670080-4a08-4c34-8105-a23249c3b65b admin admin] RBAC: Authorizing `identity:create_grant(project_id=242d586d63f3499186a3116e161b6a0e, user_id=644fa17c5d284d3a86f54840efbc6408, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:50.979518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74670080-4a08-4c34-8105-a23249c3b65b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:50.995807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1020/2024] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:50 2022] PUT /identity/v3/projects/242d586d63f3499186a3116e161b6a0e/users/644fa17c5d284d3a86f54840efbc6408/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:51.004354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:51.006211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:51.018987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611620-4936-41b5-b14b-7baa65d80516 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:51.019683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611620-4936-41b5-b14b-7baa65d80516 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:51.020271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00611620-4936-41b5-b14b-7baa65d80516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:51.028520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b44642-c1ab-4efd-bf76-babca235046b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:51.087017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:51.093011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-00611620-4936-41b5-b14b-7baa65d80516 None None] MFA Rules not processed for user `644fa17c5d284d3a86f54840efbc6408`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:51.095429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-00611620-4936-41b5-b14b-7baa65d80516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:51.198504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:51.200532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:51.201018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:51.201577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] PATH_INFO: `/v3/projects/d851da34bbd44716a65fbb0cafdf65d4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:51.203943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d851da34bbd44716a65fbb0cafdf65d4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:51.209884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:51.243745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1021/2025] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:51.266198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611620-4936-41b5-b14b-7baa65d80516 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:51.271158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-00611620-4936-41b5-b14b-7baa65d80516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:51.365499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd851da34bbd44716a65fbb0cafdf65d4', 'request_id': 'req-17247c0b-cea8-42ca-a10d-a8578e3883ea'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:51.376530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd851da34bbd44716a65fbb0cafdf65d4', 'request_id': 'req-17247c0b-cea8-42ca-a10d-a8578e3883ea'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:51.386377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd851da34bbd44716a65fbb0cafdf65d4', 'request_id': 'req-17247c0b-cea8-42ca-a10d-a8578e3883ea'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:51.416813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] The token cache is being invalidate because project d851da34bbd44716a65fbb0cafdf65d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:51.417861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:51.419532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-17247c0b-cea8-42ca-a10d-a8578e3883ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:51.423791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1005/2026] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:51 2022] DELETE /identity/v3/projects/d851da34bbd44716a65fbb0cafdf65d4 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:51.429018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00611620-4936-41b5-b14b-7baa65d80516 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:51.430897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff19608-52cc-4433-afb2-6f95cbc0b818 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:51.431398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff19608-52cc-4433-afb2-6f95cbc0b818 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:51.431907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff19608-52cc-4433-afb2-6f95cbc0b818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:51.489690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9ff19608-52cc-4433-afb2-6f95cbc0b818 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:51.492756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9ff19608-52cc-4433-afb2-6f95cbc0b818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:51.520255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:51.521190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:51.521525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:51.521833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:51.524751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:51.603759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:51.645488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1006/2027] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:51.649391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bd8d2-9cd3-4089-af43-b0b110099f7c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:51.650139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bd8d2-9cd3-4089-af43-b0b110099f7c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:51.650767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bd8d2-9cd3-4089-af43-b0b110099f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:51.664381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:51.667565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:51.687020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b77bd8d2-9cd3-4089-af43-b0b110099f7c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:51.689096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b77bd8d2-9cd3-4089-af43-b0b110099f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:51.690035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1022/2028] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:31:51.707386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1007/2029] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:51.710478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:51.721966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:51.743958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026a1ae-4350-4b76-9ce5-d6bd05f373ac service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:51.754523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:51.755515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:51.756006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:51.756440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:51.757602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:51.758147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:51.762045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:51.802111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1023/2030] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:51.803066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bd8d2-9cd3-4089-af43-b0b110099f7c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:51.822142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bd8d2-9cd3-4089-af43-b0b110099f7c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:51.834175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fd3dd-7330-432b-b5b7-c9e224199e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:51.835617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fd3dd-7330-432b-b5b7-c9e224199e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:51.836659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fd3dd-7330-432b-b5b7-c9e224199e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:51.837662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fd3dd-7330-432b-b5b7-c9e224199e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:51.839028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fd3dd-7330-432b-b5b7-c9e224199e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:51.849555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fd3dd-7330-432b-b5b7-c9e224199e29 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:51.897660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:51.901028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:51.905065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1008/2031] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:51 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:51.910973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fd3dd-7330-432b-b5b7-c9e224199e29 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:51.930739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e57bd-2371-44a1-92b3-bffffbdc40ca admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:52.054794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:52.055594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:52.055954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:52.056282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:52.061373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:52.071122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fd3dd-7330-432b-b5b7-c9e224199e29 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:52.077743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8219b97-7225-4f72-bc67-7b53977fb485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:52.078476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219b97-7225-4f72-bc67-7b53977fb485 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:52.078821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219b97-7225-4f72-bc67-7b53977fb485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:52.079177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8219b97-7225-4f72-bc67-7b53977fb485 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:52.080416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8219b97-7225-4f72-bc67-7b53977fb485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:52.082596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8219b97-7225-4f72-bc67-7b53977fb485 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:52.112757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1009/2032] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:51 2022] POST /identity/v3/users => generated 284 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:52.118697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8219b97-7225-4f72-bc67-7b53977fb485 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:52.132340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8219b97-7225-4f72-bc67-7b53977fb485 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:52.138544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336e287-ffce-4b10-976b-beae245c1607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:52.139270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336e287-ffce-4b10-976b-beae245c1607 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:52.139634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336e287-ffce-4b10-976b-beae245c1607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:52.139915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336e287-ffce-4b10-976b-beae245c1607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:52.140566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336e287-ffce-4b10-976b-beae245c1607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:52.142264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336e287-ffce-4b10-976b-beae245c1607 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:52.154601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1010/2033] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:31:52.159656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336e287-ffce-4b10-976b-beae245c1607 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:52.172377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:52.173632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336e287-ffce-4b10-976b-beae245c1607 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:52.175585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:52.179729 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed77f4-5cb8-493e-8335-954f894f46ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:52.180249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed77f4-5cb8-493e-8335-954f894f46ce admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:52.180768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed77f4-5cb8-493e-8335-954f894f46ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:52.180998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed77f4-5cb8-493e-8335-954f894f46ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:52.181788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed77f4-5cb8-493e-8335-954f894f46ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:52.183714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed77f4-5cb8-493e-8335-954f894f46ce admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:52.193879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1024/2034] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:52.210358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1011/2035] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:31:52.219460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:52.241768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0622c-3950-4a2b-ae1c-a6f7f23e75b5 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:52.254859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9feb478-bef4-4c18-8837-3cb828f3c719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:52.256097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb478-bef4-4c18-8837-3cb828f3c719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:52.256668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb478-bef4-4c18-8837-3cb828f3c719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:52.257108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9feb478-bef4-4c18-8837-3cb828f3c719 admin admin] PATH_INFO: `/v3/projects/7bebec24f87e4fea8e99dbbb99717d61/users/d96425b0b37b47d5bf9ecddc070d79d4/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:52.288119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9feb478-bef4-4c18-8837-3cb828f3c719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bebec24f87e4fea8e99dbbb99717d61, user_id=d96425b0b37b47d5bf9ecddc070d79d4, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:52.293561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9feb478-bef4-4c18-8837-3cb828f3c719 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:52.318473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1025/2036] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:52 2022] PUT /identity/v3/projects/7bebec24f87e4fea8e99dbbb99717d61/users/d96425b0b37b47d5bf9ecddc070d79d4/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:52.337833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822616-de60-4d8d-b83f-b2cfbb34d9ee None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:52.338320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822616-de60-4d8d-b83f-b2cfbb34d9ee None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:52.338826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822616-de60-4d8d-b83f-b2cfbb34d9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:52.399702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-af822616-de60-4d8d-b83f-b2cfbb34d9ee None None] MFA Rules not processed for user `d96425b0b37b47d5bf9ecddc070d79d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:52.401214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-af822616-de60-4d8d-b83f-b2cfbb34d9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:52.588894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1012/2037] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:52.597027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9feb478-bef4-4c18-8837-3cb828f3c719 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:52.926249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9feb478-bef4-4c18-8837-3cb828f3c719 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:52.944850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4733eb-54aa-4909-86dc-cc3d49603aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:52.944850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4733eb-54aa-4909-86dc-cc3d49603aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:52.946895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4733eb-54aa-4909-86dc-cc3d49603aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:52.954953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4733eb-54aa-4909-86dc-cc3d49603aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:52.960761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4733eb-54aa-4909-86dc-cc3d49603aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:52.976362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4733eb-54aa-4909-86dc-cc3d49603aa6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:53.078821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1026/2038] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:52 2022] POST /identity/v3/projects => generated 367 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:53.083308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822616-de60-4d8d-b83f-b2cfbb34d9ee None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:53.129400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822616-de60-4d8d-b83f-b2cfbb34d9ee None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:53.183183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:53.259374 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16362218-3138-4e21-ae32-6e3061a44655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:53.260387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16362218-3138-4e21-ae32-6e3061a44655 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:53.260551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16362218-3138-4e21-ae32-6e3061a44655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:53.261044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16362218-3138-4e21-ae32-6e3061a44655 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:53.261738 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16362218-3138-4e21-ae32-6e3061a44655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:53.264568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16362218-3138-4e21-ae32-6e3061a44655 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:53.327492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1013/2039] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:53 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:31:53.331588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4733eb-54aa-4909-86dc-cc3d49603aa6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:53.352536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4733eb-54aa-4909-86dc-cc3d49603aa6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:53.361916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:53.362574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:53.362858 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:53.363139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:53.363835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:53.365866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:53.368808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:53.391525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1027/2040] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:31:53.397943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16362218-3138-4e21-ae32-6e3061a44655 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:53.415096 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16362218-3138-4e21-ae32-6e3061a44655 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:53.423148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a5648-f607-47be-8211-d92176684afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:53.425037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5648-f607-47be-8211-d92176684afc admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:53.426041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5648-f607-47be-8211-d92176684afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:53.426526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a5648-f607-47be-8211-d92176684afc admin admin] PATH_INFO: `/v3/projects/813f46e0bc5641589b15c33dc7a7e674/users/7c443090e773446faab72ad0b225f905/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:53.448565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:53.452070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:53.463687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a5648-f607-47be-8211-d92176684afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=813f46e0bc5641589b15c33dc7a7e674, user_id=7c443090e773446faab72ad0b225f905, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:53.467394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a5648-f607-47be-8211-d92176684afc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:53.490148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1014/2041] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:53 2022] PUT /identity/v3/projects/813f46e0bc5641589b15c33dc7a7e674/users/7c443090e773446faab72ad0b225f905/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:53.491277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a5648-f607-47be-8211-d92176684afc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:53.612420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b5049-bc93-4f2f-aebc-310d0caaa2f9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:53.619582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a5648-f607-47be-8211-d92176684afc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:53.628924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984ab57-f36d-4373-868c-6794de6655a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:53.630004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ab57-f36d-4373-868c-6794de6655a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:53.630556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ab57-f36d-4373-868c-6794de6655a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:53.631002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984ab57-f36d-4373-868c-6794de6655a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:53.632243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984ab57-f36d-4373-868c-6794de6655a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:53.635528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984ab57-f36d-4373-868c-6794de6655a1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:53.665422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1015/2042] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:31:53.670821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984ab57-f36d-4373-868c-6794de6655a1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:53.683045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984ab57-f36d-4373-868c-6794de6655a1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:53.690340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d336863-276c-4671-a763-c49ae3bded7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:53.691175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d336863-276c-4671-a763-c49ae3bded7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:53.691470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d336863-276c-4671-a763-c49ae3bded7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:53.691749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d336863-276c-4671-a763-c49ae3bded7e admin admin] PATH_INFO: `/v3/projects/813f46e0bc5641589b15c33dc7a7e674/users/7c443090e773446faab72ad0b225f905/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:53.693356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:53.693864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d336863-276c-4671-a763-c49ae3bded7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=813f46e0bc5641589b15c33dc7a7e674, user_id=7c443090e773446faab72ad0b225f905, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:53.694291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:53.694677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:53.695209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:53.697087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d336863-276c-4671-a763-c49ae3bded7e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:53.697722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:53.722606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1016/2043] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:31:53 2022] PUT /identity/v3/projects/813f46e0bc5641589b15c33dc7a7e674/users/7c443090e773446faab72ad0b225f905/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:53.730345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d1167-20f3-4753-bfb7-0b7d7ef585c6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:53.730652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d1167-20f3-4753-bfb7-0b7d7ef585c6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:53.731470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d1167-20f3-4753-bfb7-0b7d7ef585c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:53.770265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-796d1167-20f3-4753-bfb7-0b7d7ef585c6 None None] MFA Rules not processed for user `7c443090e773446faab72ad0b225f905`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:53.771709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-796d1167-20f3-4753-bfb7-0b7d7ef585c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:53.821834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:53.826505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:53.839237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1028/2044] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:53.928388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:53.952956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1017/2045] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:54.023684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:54.042833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d075778-1e69-4e0a-9412-aa855225bd9d service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:54.158849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:54.159917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:54.160452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:54.161026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:54.177440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:54.179585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:54.192331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1029/2046] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:55.165009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d1167-20f3-4753-bfb7-0b7d7ef585c6 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:55.181983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d1167-20f3-4753-bfb7-0b7d7ef585c6 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:55.201153 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad6ddf-7d79-4ce4-957a-4bf0072a71a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:55.202057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad6ddf-7d79-4ce4-957a-4bf0072a71a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:55.202427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad6ddf-7d79-4ce4-957a-4bf0072a71a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:55.202596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad6ddf-7d79-4ce4-957a-4bf0072a71a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:55.205939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-07ad6ddf-7d79-4ce4-957a-4bf0072a71a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:55.293326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad6ddf-7d79-4ce4-957a-4bf0072a71a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:55.294955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad6ddf-7d79-4ce4-957a-4bf0072a71a2 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:55.313114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1018/2047] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:56.491962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:56.494213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:56.508890 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:56.581934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f13251-fcc9-48fc-9605-5b5e75c5611d service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:56.598372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:56.599197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:56.599655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:56.599815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] PATH_INFO: `/v3/users/05037938f5844812a55be6e77da8ab60` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:56.601055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05037938f5844812a55be6e77da8ab60)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:56.603127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:56.683244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05037938f5844812a55be6e77da8ab60'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:56.720497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05037938f5844812a55be6e77da8ab60'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:56.721969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05037938f5844812a55be6e77da8ab60'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:56.739137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05037938f5844812a55be6e77da8ab60'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:56.754075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05037938f5844812a55be6e77da8ab60'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:56.765406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1030/2048] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:56 2022] DELETE /identity/v3/users/05037938f5844812a55be6e77da8ab60 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:56.771790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327ad0-03fc-448d-8ed6-5b73ca8fcc1c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:56.772813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327ad0-03fc-448d-8ed6-5b73ca8fcc1c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:56.773264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0327ad0-03fc-448d-8ed6-5b73ca8fcc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:56.804879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e0327ad0-03fc-448d-8ed6-5b73ca8fcc1c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:56.807463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e0327ad0-03fc-448d-8ed6-5b73ca8fcc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:56.938847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1019/2049] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:56.955289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:56.957432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:57.180807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214d717-cc6c-4d4c-88d2-259146da1d49 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:57.190185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a8e3a-5f97-4a20-a30e-6db6f21793af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:57.190943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a8e3a-5f97-4a20-a30e-6db6f21793af service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:57.191231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a8e3a-5f97-4a20-a30e-6db6f21793af service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:57.191826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a8e3a-5f97-4a20-a30e-6db6f21793af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:57.218590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a8e3a-5f97-4a20-a30e-6db6f21793af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:57.220634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a8e3a-5f97-4a20-a30e-6db6f21793af service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:57.243450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1031/2050] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:31:57.795838 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327ad0-03fc-448d-8ed6-5b73ca8fcc1c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:57.798642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e0327ad0-03fc-448d-8ed6-5b73ca8fcc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:57.841938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0327ad0-03fc-448d-8ed6-5b73ca8fcc1c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:57.861767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:57.863008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:57.863509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:57.863983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] PATH_INFO: `/v3/projects/22fa6c08ed81470b86bdbab4c38c8fcb` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:57.869229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] RBAC: Authorizing `identity:delete_project(project_id=22fa6c08ed81470b86bdbab4c38c8fcb)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:57.875009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:57.948738 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22fa6c08ed81470b86bdbab4c38c8fcb', 'request_id': 'req-fdf1afbd-2897-415b-96ad-2cbf0a7abced'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:57.970298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22fa6c08ed81470b86bdbab4c38c8fcb', 'request_id': 'req-fdf1afbd-2897-415b-96ad-2cbf0a7abced'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:57.983870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22fa6c08ed81470b86bdbab4c38c8fcb', 'request_id': 'req-fdf1afbd-2897-415b-96ad-2cbf0a7abced'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.010796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] The token cache is being invalidate because project 22fa6c08ed81470b86bdbab4c38c8fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:58.011427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.013718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.016534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1020/2051] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:57 2022] DELETE /identity/v3/projects/22fa6c08ed81470b86bdbab4c38c8fcb => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:58.022021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a8e3a-5f97-4a20-a30e-6db6f21793af service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:58.037408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a8e3a-5f97-4a20-a30e-6db6f21793af service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:58.076303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:58.079263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:58.081554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:58.106314 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:58.136430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:58.138031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:58.138989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:58.139270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] PATH_INFO: `/v3/users/644fa17c5d284d3a86f54840efbc6408` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:58.140950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644fa17c5d284d3a86f54840efbc6408)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:58.143469 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:58.239499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644fa17c5d284d3a86f54840efbc6408'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.244621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1afbd-2897-415b-96ad-2cbf0a7abced admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:58.254156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:58.255231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:58.255622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:58.256320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] PATH_INFO: `/v3/users/d96425b0b37b47d5bf9ecddc070d79d4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:58.258895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96425b0b37b47d5bf9ecddc070d79d4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:58.259975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644fa17c5d284d3a86f54840efbc6408'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.261504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:58.262618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644fa17c5d284d3a86f54840efbc6408'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.294576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644fa17c5d284d3a86f54840efbc6408'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.321782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a9db0a39-4084-4b8b-9d55-89113db0c9d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644fa17c5d284d3a86f54840efbc6408'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.335566 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1032/2052] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:58 2022] DELETE /identity/v3/users/644fa17c5d284d3a86f54840efbc6408 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:58.392152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96425b0b37b47d5bf9ecddc070d79d4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.393270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96425b0b37b47d5bf9ecddc070d79d4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.397785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96425b0b37b47d5bf9ecddc070d79d4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.417146 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96425b0b37b47d5bf9ecddc070d79d4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.451808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96425b0b37b47d5bf9ecddc070d79d4'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.483021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1021/2053] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:58 2022] DELETE /identity/v3/users/d96425b0b37b47d5bf9ecddc070d79d4 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:58.490043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8f1b0-3ba1-4bf7-b426-340cec50c72e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:58.490463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8f1b0-3ba1-4bf7-b426-340cec50c72e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:58.490860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8f1b0-3ba1-4bf7-b426-340cec50c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:58.514806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3ac8f1b0-3ba1-4bf7-b426-340cec50c72e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:58.516504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3ac8f1b0-3ba1-4bf7-b426-340cec50c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:58.630215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1033/2054] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:58.646223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:58.649763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:58.702278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8f1b0-3ba1-4bf7-b426-340cec50c72e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:58.702866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3ac8f1b0-3ba1-4bf7-b426-340cec50c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:58.726809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8f1b0-3ba1-4bf7-b426-340cec50c72e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:58.744976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:58.745685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:58.746098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:58.746242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] PATH_INFO: `/v3/users/61661d1fe90d4d46b59ae09c178a4c9d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:58.749299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61661d1fe90d4d46b59ae09c178a4c9d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:58.753616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:58.860604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d046d4-a443-4cb6-8419-4cbfb1d30896 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:58.873030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:58.878675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61661d1fe90d4d46b59ae09c178a4c9d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.892148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:58.893276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:58.893811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:58.894792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:58.912719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61661d1fe90d4d46b59ae09c178a4c9d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.914197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61661d1fe90d4d46b59ae09c178a4c9d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.935654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61661d1fe90d4d46b59ae09c178a4c9d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.943351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:58.948101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:58.956182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ca890fbd-09b2-4f44-a895-1ec86a1aae91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61661d1fe90d4d46b59ae09c178a4c9d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:58.977817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1034/2055] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:31:58 2022] DELETE /identity/v3/users/61661d1fe90d4d46b59ae09c178a4c9d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:31:58.981732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1022/2056] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:31:58.988132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804d08c-a677-447b-9a7b-6b0618eb5e5b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:58.988435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804d08c-a677-447b-9a7b-6b0618eb5e5b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:58.988724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804d08c-a677-447b-9a7b-6b0618eb5e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.023190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a804d08c-a677-447b-9a7b-6b0618eb5e5b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:59.024805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a804d08c-a677-447b-9a7b-6b0618eb5e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:59.093623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:59.096007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:59.117777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699f461-9d89-4e1c-8793-05097be76190 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:59.173812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1035/2057] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:59.187597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804d08c-a677-447b-9a7b-6b0618eb5e5b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:59.190536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a804d08c-a677-447b-9a7b-6b0618eb5e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:59.221955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:59.271156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:59.272526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.272826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.273146 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] PATH_INFO: `/v3/projects/242d586d63f3499186a3116e161b6a0e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.274627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242d586d63f3499186a3116e161b6a0e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:59.277481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:59.318018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804d08c-a677-447b-9a7b-6b0618eb5e5b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:59.325343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4b00f-c75a-4c05-8d2d-72d85da0d720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:59.326298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4b00f-c75a-4c05-8d2d-72d85da0d720 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.326819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4b00f-c75a-4c05-8d2d-72d85da0d720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.327566 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4b00f-c75a-4c05-8d2d-72d85da0d720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.354878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4b00f-c75a-4c05-8d2d-72d85da0d720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:59.357785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4b00f-c75a-4c05-8d2d-72d85da0d720 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:59.366986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242d586d63f3499186a3116e161b6a0e', 'request_id': 'req-d5c74de7-a8f1-442e-a8e8-1029397b42f9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.375520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242d586d63f3499186a3116e161b6a0e', 'request_id': 'req-d5c74de7-a8f1-442e-a8e8-1029397b42f9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.379553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1036/2058] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:31:59.391524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242d586d63f3499186a3116e161b6a0e', 'request_id': 'req-d5c74de7-a8f1-442e-a8e8-1029397b42f9'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.423541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] The token cache is being invalidate because project 242d586d63f3499186a3116e161b6a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:59.424107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.426071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.430849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1023/2059] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:59 2022] DELETE /identity/v3/projects/242d586d63f3499186a3116e161b6a0e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:59.437358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527bca8-548b-4baf-9822-ab49f5403687 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.437903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527bca8-548b-4baf-9822-ab49f5403687 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.438396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527bca8-548b-4baf-9822-ab49f5403687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.477421 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d527bca8-548b-4baf-9822-ab49f5403687 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:59.479684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d527bca8-548b-4baf-9822-ab49f5403687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:59.535369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:59.539773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:59.616161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1037/2060] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:31:59.625341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50b003-d284-4bb4-9977-1d48be2da41f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.625745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50b003-d284-4bb4-9977-1d48be2da41f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.626105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50b003-d284-4bb4-9977-1d48be2da41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.646916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:59.654277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5c50b003-d284-4bb4-9977-1d48be2da41f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:31:59.655751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5c50b003-d284-4bb4-9977-1d48be2da41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:59.675085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1038/2061] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:31:59.680538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50b003-d284-4bb4-9977-1d48be2da41f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:59.695699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c74de7-a8f1-442e-a8e8-1029397b42f9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:59.704508 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:59.704719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50b003-d284-4bb4-9977-1d48be2da41f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:59.705562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.705959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.707709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] PATH_INFO: `/v3/projects/7bebec24f87e4fea8e99dbbb99717d61` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.710165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bebec24f87e4fea8e99dbbb99717d61)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:59.713119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:59.713296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:59.714368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.714849 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.715470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.717078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:31:59.717655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:59.721192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:59.752567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1039/2062] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:31:59.756568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:59.770051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3014eb2-1b65-4282-bc51-d7cd71751832 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:59.776426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:59.777013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.777301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.778508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.778957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:59.781285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:59.820664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bebec24f87e4fea8e99dbbb99717d61', 'request_id': 'req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.829227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bebec24f87e4fea8e99dbbb99717d61', 'request_id': 'req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.829786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1040/2063] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:31:59 2022] POST /identity/v3/projects => generated 407 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:31:59.830579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:59.832779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:59.837691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bebec24f87e4fea8e99dbbb99717d61', 'request_id': 'req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.842295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:31:59.858235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] The token cache is being invalidate because project 7bebec24f87e4fea8e99dbbb99717d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:31:59.858757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.860463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.862459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1024/2064] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:59 2022] DELETE /identity/v3/projects/7bebec24f87e4fea8e99dbbb99717d61 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:31:59.863278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:31:59.866334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:31:59.912620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d94175-464a-4760-89ce-c26b3a908f92 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:59.918647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:59.918647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.919269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.919560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] PATH_INFO: `/v3/projects/6986cb28eb3f47a5a001115df4c1e4b7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.920876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6986cb28eb3f47a5a001115df4c1e4b7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:59.922863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:59.952610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74a5dd6-3788-47e9-a1f7-84cad68bb7d5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:31:59.966481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:31:59.968145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:31:59.968644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:31:59.969124 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:31:59.970362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:31:59.974695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:31:59.986747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6986cb28eb3f47a5a001115df4c1e4b7', 'request_id': 'req-ff51588b-81a6-447c-a71f-2f1561da98dd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:31:59.993921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6986cb28eb3f47a5a001115df4c1e4b7', 'request_id': 'req-ff51588b-81a6-447c-a71f-2f1561da98dd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.008107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6986cb28eb3f47a5a001115df4c1e4b7', 'request_id': 'req-ff51588b-81a6-447c-a71f-2f1561da98dd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.018114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1025/2065] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:31:59 2022] POST /identity/v3/users => generated 304 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:00.018114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:00.020139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.030487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] The token cache is being invalidate because project 6986cb28eb3f47a5a001115df4c1e4b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:00.030908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.031524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.033049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1041/2066] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:31:59 2022] DELETE /identity/v3/projects/6986cb28eb3f47a5a001115df4c1e4b7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:00.033897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:00.035510 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.042819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bb837-2d2c-471f-869c-7034d0356229 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:00.149935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:00.160900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:00.161399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:00.161885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:00.162051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] PATH_INFO: `/v3/users/7c443090e773446faab72ad0b225f905` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:00.163478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c443090e773446faab72ad0b225f905)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:00.164238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51588b-81a6-447c-a71f-2f1561da98dd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:00.165521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:00.183120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:00.185002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:00.185002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:00.185002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:00.185763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:00.187372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:00.201846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1042/2067] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:00.205937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:00.208243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.235838 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c443090e773446faab72ad0b225f905'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.236868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c443090e773446faab72ad0b225f905'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.237538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe23ff-1ab8-4714-8da2-0d67ad161d0a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:00.241659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c443090e773446faab72ad0b225f905'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.248194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:00.248892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:00.249243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:00.249595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] PATH_INFO: `/v3/users/f937c6752683410aba1570ea12c53377` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:00.251181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f937c6752683410aba1570ea12c53377)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:00.253536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:00.253809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c443090e773446faab72ad0b225f905'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.274037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c443090e773446faab72ad0b225f905'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.297817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1026/2068] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:00 2022] DELETE /identity/v3/users/7c443090e773446faab72ad0b225f905 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:00.299712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:00.302891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.316071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:00.346807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f937c6752683410aba1570ea12c53377'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.397688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f937c6752683410aba1570ea12c53377'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.399573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f937c6752683410aba1570ea12c53377'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.424012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f937c6752683410aba1570ea12c53377'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.444275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f937c6752683410aba1570ea12c53377'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.458716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1043/2069] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:00 2022] DELETE /identity/v3/users/f937c6752683410aba1570ea12c53377 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:00.459907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:00.464851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.472471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921aa99-5080-40ab-8ab0-b62be0db32c2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:00.578236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:00.579236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:00.579703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:00.580260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:00.581083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:00.582947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:00.606228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1027/2070] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:00.607052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ecf99-2449-4454-95fd-1b1d4f1f9f9e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:00.613321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:00.613921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:00.614075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:00.614159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:00.614429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] PATH_INFO: `/v3/users/900de1d2272d4ffb8c0d60163ee5e5b8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:00.615789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=900de1d2272d4ffb8c0d60163ee5e5b8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:00.617899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:00.630476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.659667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7a08a-6d2e-4742-bb15-1342d39ca27c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:00.667682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c9a6f-bfa6-41f4-9e57-6ff2b1f0162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:00.670312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c9a6f-bfa6-41f4-9e57-6ff2b1f0162c admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:00.671196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c9a6f-bfa6-41f4-9e57-6ff2b1f0162c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:00.671832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c9a6f-bfa6-41f4-9e57-6ff2b1f0162c admin admin] PATH_INFO: `/v3/projects/b4e42e3a5abe462c80c6fd12973618a3/users/7c3cbb6c23e2487e8d798e87f8310989/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:00.696728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c9a6f-bfa6-41f4-9e57-6ff2b1f0162c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e42e3a5abe462c80c6fd12973618a3, user_id=7c3cbb6c23e2487e8d798e87f8310989, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:00.698609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900de1d2272d4ffb8c0d60163ee5e5b8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.701044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c9a6f-bfa6-41f4-9e57-6ff2b1f0162c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:00.725307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1028/2071] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:00 2022] PUT /identity/v3/projects/b4e42e3a5abe462c80c6fd12973618a3/users/7c3cbb6c23e2487e8d798e87f8310989/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:00.736408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900de1d2272d4ffb8c0d60163ee5e5b8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.737360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900de1d2272d4ffb8c0d60163ee5e5b8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.737695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd330ba-d36e-4e34-a93f-955857c98349 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:00.737995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd330ba-d36e-4e34-a93f-955857c98349 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:00.738336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd330ba-d36e-4e34-a93f-955857c98349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:00.739752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:00.750208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900de1d2272d4ffb8c0d60163ee5e5b8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.766762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e6c70c1-c6bc-46db-9c8b-11d12cc98020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900de1d2272d4ffb8c0d60163ee5e5b8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:00.778038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1044/2072] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:00 2022] DELETE /identity/v3/users/900de1d2272d4ffb8c0d60163ee5e5b8 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:00.782675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9830d38-661e-4e13-86d8-d51806450649 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:00.783161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9830d38-661e-4e13-86d8-d51806450649 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:00.784259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9830d38-661e-4e13-86d8-d51806450649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:00.788357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5bd330ba-d36e-4e34-a93f-955857c98349 None None] MFA Rules not processed for user `7c3cbb6c23e2487e8d798e87f8310989`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:00.789698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5bd330ba-d36e-4e34-a93f-955857c98349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.829051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a9830d38-661e-4e13-86d8-d51806450649 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:00.831138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a9830d38-661e-4e13-86d8-d51806450649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.941648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1029/2073] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:00 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:00.957157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd330ba-d36e-4e34-a93f-955857c98349 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:00.964162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5bd330ba-d36e-4e34-a93f-955857c98349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:00.998358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:01.002290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1045/2074] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:01.010939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9830d38-661e-4e13-86d8-d51806450649 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:01.015203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a9830d38-661e-4e13-86d8-d51806450649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.040161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9830d38-661e-4e13-86d8-d51806450649 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:01.047633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:01.048326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.048614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.048898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] PATH_INFO: `/v3/projects/813f46e0bc5641589b15c33dc7a7e674` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.050960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=813f46e0bc5641589b15c33dc7a7e674)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:01.053383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:01.089294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd330ba-d36e-4e34-a93f-955857c98349 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:01.096807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:01.098544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.098544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.098544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.099274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:01.102168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:01.145534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813f46e0bc5641589b15c33dc7a7e674', 'request_id': 'req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.145819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1030/2075] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:00 2022] POST /identity/v3/projects => generated 407 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:01.145819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:01.147140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.149207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813f46e0bc5641589b15c33dc7a7e674', 'request_id': 'req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.164292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813f46e0bc5641589b15c33dc7a7e674', 'request_id': 'req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.179939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] The token cache is being invalidate because project 813f46e0bc5641589b15c33dc7a7e674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:01.180353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.181094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.190080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1046/2076] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:01 2022] DELETE /identity/v3/projects/813f46e0bc5641589b15c33dc7a7e674 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:01.190080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:01.190080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.263803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c134d1-f5c2-424d-b6d6-ae65a893ffc5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:01.271630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0964bdf-d76f-4bc4-99db-ec541e871c30 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:01.278877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:01.279921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.280442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.281123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] PATH_INFO: `/v3/projects/c723600975514a77a647b0953eb7ad8c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.282613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c723600975514a77a647b0953eb7ad8c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:01.285140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:01.289439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:01.329716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:01.330431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.330717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.331023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.339241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.347930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c723600975514a77a647b0953eb7ad8c', 'request_id': 'req-b305fb2b-13fe-4542-9d20-0fffac98b777'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.356591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:01.357759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c723600975514a77a647b0953eb7ad8c', 'request_id': 'req-b305fb2b-13fe-4542-9d20-0fffac98b777'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.358542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:01.364540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c723600975514a77a647b0953eb7ad8c', 'request_id': 'req-b305fb2b-13fe-4542-9d20-0fffac98b777'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.375890 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1047/2077] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:01.377549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:01.379810 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.381046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] The token cache is being invalidate because project c723600975514a77a647b0953eb7ad8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:01.381333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.382067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b305fb2b-13fe-4542-9d20-0fffac98b777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:01.383882 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1031/2078] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:01 2022] DELETE /identity/v3/projects/c723600975514a77a647b0953eb7ad8c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:01.385294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ee6af-81c6-47a3-a75f-14bd4781c449 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.385665 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ee6af-81c6-47a3-a75f-14bd4781c449 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.386006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ee6af-81c6-47a3-a75f-14bd4781c449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.401102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c4c0d3-4d7a-4689-b2ea-1c3da99f5c17 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:01.409605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9b2ee6af-81c6-47a3-a75f-14bd4781c449 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:01.411365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9b2ee6af-81c6-47a3-a75f-14bd4781c449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.438628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:01.520564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200f32f-4c02-4705-bff4-c9316df206c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:01.521648 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200f32f-4c02-4705-bff4-c9316df206c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.522131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200f32f-4c02-4705-bff4-c9316df206c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.522592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200f32f-4c02-4705-bff4-c9316df206c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.523747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200f32f-4c02-4705-bff4-c9316df206c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:01.527323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1032/2079] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:01.530301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9382-4972-4867-a604-0fc177fc1394 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.530930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9382-4972-4867-a604-0fc177fc1394 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.531706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9382-4972-4867-a604-0fc177fc1394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.537977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200f32f-4c02-4705-bff4-c9316df206c8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:01.577609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fe4f9382-4972-4867-a604-0fc177fc1394 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:01.579690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fe4f9382-4972-4867-a604-0fc177fc1394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.581871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1048/2080] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:01 2022] POST /identity/v3/users => generated 310 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:01.584295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58656ef-7a2f-4354-86d4-7f91f07d9028 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.584800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58656ef-7a2f-4354-86d4-7f91f07d9028 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.586328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58656ef-7a2f-4354-86d4-7f91f07d9028 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.588923 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1049/2081] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:32:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:32:01.592075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5b08-3a88-4914-9543-b0e073cf9185 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.592574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5b08-3a88-4914-9543-b0e073cf9185 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.592964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c5b08-3a88-4914-9543-b0e073cf9185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.606660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1033/2082] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:01.606660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f9382-4972-4867-a604-0fc177fc1394 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:01.606660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fe4f9382-4972-4867-a604-0fc177fc1394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.623399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f9382-4972-4867-a604-0fc177fc1394 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:01.630102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ff799-9ab5-45d1-8d46-5e7a52f49e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:01.630760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff799-9ab5-45d1-8d46-5e7a52f49e28 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.631050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff799-9ab5-45d1-8d46-5e7a52f49e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.631430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff799-9ab5-45d1-8d46-5e7a52f49e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.632796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ff799-9ab5-45d1-8d46-5e7a52f49e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:01.634531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ff799-9ab5-45d1-8d46-5e7a52f49e28 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:01.644022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-016c5b08-3a88-4914-9543-b0e073cf9185 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:01.646488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-016c5b08-3a88-4914-9543-b0e073cf9185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.651482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1034/2083] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:01.653535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4613b-a29c-4e50-945b-d9ed4722af12 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.653833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4613b-a29c-4e50-945b-d9ed4722af12 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.654109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4613b-a29c-4e50-945b-d9ed4722af12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.677913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1050/2084] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:01.690156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063bdb7-3b05-4f2b-9026-8164395a1314 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.690935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063bdb7-3b05-4f2b-9026-8164395a1314 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.691501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063bdb7-3b05-4f2b-9026-8164395a1314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.698439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d3e4613b-a29c-4e50-945b-d9ed4722af12 None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:01.701630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d3e4613b-a29c-4e50-945b-d9ed4722af12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.742787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1063bdb7-3b05-4f2b-9026-8164395a1314 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:01.746168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1063bdb7-3b05-4f2b-9026-8164395a1314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:01.781966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1051/2085] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:01.783860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063bdb7-3b05-4f2b-9026-8164395a1314 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:01.803323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063bdb7-3b05-4f2b-9026-8164395a1314 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:01.808463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38031ca-b8bb-4e3f-a40d-3ce745b33741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:01.809786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38031ca-b8bb-4e3f-a40d-3ce745b33741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:01.810252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38031ca-b8bb-4e3f-a40d-3ce745b33741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:01.810536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38031ca-b8bb-4e3f-a40d-3ce745b33741 admin admin] PATH_INFO: `/v3/projects/3836cab5341d4a6fbd53970562a58819/users/a28a0c41c43246afa22643b472e70cf7/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:01.832017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38031ca-b8bb-4e3f-a40d-3ce745b33741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3836cab5341d4a6fbd53970562a58819, user_id=a28a0c41c43246afa22643b472e70cf7, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:01.835410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38031ca-b8bb-4e3f-a40d-3ce745b33741 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:01.862250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1052/2086] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:01 2022] PUT /identity/v3/projects/3836cab5341d4a6fbd53970562a58819/users/a28a0c41c43246afa22643b472e70cf7/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:01.862250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38031ca-b8bb-4e3f-a40d-3ce745b33741 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:01.886336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1035/2087] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:32:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:01.889509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4613b-a29c-4e50-945b-d9ed4722af12 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:01.908486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4613b-a29c-4e50-945b-d9ed4722af12 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:01.977324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38031ca-b8bb-4e3f-a40d-3ce745b33741 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.000305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.001500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.003612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.004476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.005685 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.006420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:02.006420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.009170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.011239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.012783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.015717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:02.017129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.020118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.022050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.053965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1036/2088] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:02.053965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.055998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1053/2089] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:02.058504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.063150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:02.071166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d8962-216c-48f4-a691-73a4421162e8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.079288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da735-5b96-481c-a6aa-b5627550057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.081090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da735-5b96-481c-a6aa-b5627550057e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.081282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da735-5b96-481c-a6aa-b5627550057e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.082542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da735-5b96-481c-a6aa-b5627550057e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.083343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149da735-5b96-481c-a6aa-b5627550057e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.086029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149da735-5b96-481c-a6aa-b5627550057e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.097991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:02.127271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1037/2090] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:02.129762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da735-5b96-481c-a6aa-b5627550057e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.149484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149da735-5b96-481c-a6aa-b5627550057e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.159026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d9939-d54a-4e68-82b0-41cabc8d94e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.159737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d9939-d54a-4e68-82b0-41cabc8d94e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.160086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d9939-d54a-4e68-82b0-41cabc8d94e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.160402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d9939-d54a-4e68-82b0-41cabc8d94e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.161264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d9939-d54a-4e68-82b0-41cabc8d94e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.164164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d9939-d54a-4e68-82b0-41cabc8d94e6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.212936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1038/2091] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:02 2022] POST /identity/v3/projects => generated 389 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:02.218383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d9939-d54a-4e68-82b0-41cabc8d94e6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.240276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:02.280979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd4722-8af6-4b5f-ae1c-4bf02e4a21a3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.368020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d9939-d54a-4e68-82b0-41cabc8d94e6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.377060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe32087-48b1-4a4d-8855-3bab502d3d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.378458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe32087-48b1-4a4d-8855-3bab502d3d6e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.378953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe32087-48b1-4a4d-8855-3bab502d3d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.379450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe32087-48b1-4a4d-8855-3bab502d3d6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.380653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe32087-48b1-4a4d-8855-3bab502d3d6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.385059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe32087-48b1-4a4d-8855-3bab502d3d6e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.398649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b0a14-4817-4597-bbd8-f2d468178208 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.399878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b0a14-4817-4597-bbd8-f2d468178208 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.399878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b0a14-4817-4597-bbd8-f2d468178208 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.400059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b0a14-4817-4597-bbd8-f2d468178208 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.411456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b0a14-4817-4597-bbd8-f2d468178208 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.413220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b0a14-4817-4597-bbd8-f2d468178208 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.432087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1039/2092] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:02 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:02.434344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe32087-48b1-4a4d-8855-3bab502d3d6e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.439035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1054/2093] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:02.441021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b0a14-4817-4597-bbd8-f2d468178208 service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.466028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b0a14-4817-4597-bbd8-f2d468178208 service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.564191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe32087-48b1-4a4d-8855-3bab502d3d6e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.573844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.575031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.575545 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.576040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] PATH_INFO: `/v3/projects/3836cab5341d4a6fbd53970562a58819/users/a28a0c41c43246afa22643b472e70cf7/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.578339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fa2a8-1e82-4518-9649-93c741b8a140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.579066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa2a8-1e82-4518-9649-93c741b8a140 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.579408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa2a8-1e82-4518-9649-93c741b8a140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.579767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fa2a8-1e82-4518-9649-93c741b8a140 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.580453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3836cab5341d4a6fbd53970562a58819, user_id=a28a0c41c43246afa22643b472e70cf7, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.580638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fa2a8-1e82-4518-9649-93c741b8a140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.582714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fa2a8-1e82-4518-9649-93c741b8a140 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.587275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.601926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1040/2094] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:02 2022] PUT /identity/v3/projects/3836cab5341d4a6fbd53970562a58819/users/a28a0c41c43246afa22643b472e70cf7/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:02.603248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.613561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:02.613833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1055/2095] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:02 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:02.614035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fa2a8-1e82-4518-9649-93c741b8a140 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.630255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fa2a8-1e82-4518-9649-93c741b8a140 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.680983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:02.697236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9025ecd-45cb-4f68-9bd0-9741f99a047e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.703379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.704053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.704336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.704705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] PATH_INFO: `/v3/projects/9935db9afd624fbb9c5a1da9273d60ed` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.706394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9935db9afd624fbb9c5a1da9273d60ed)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.708558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.739494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe079c1-3346-4428-8a79-c4e04940eb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.740128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe079c1-3346-4428-8a79-c4e04940eb02 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.740412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe079c1-3346-4428-8a79-c4e04940eb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.740689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe079c1-3346-4428-8a79-c4e04940eb02 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.741345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe079c1-3346-4428-8a79-c4e04940eb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.743452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe079c1-3346-4428-8a79-c4e04940eb02 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.757605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:02.785912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1056/2096] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:02 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:02.791960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe079c1-3346-4428-8a79-c4e04940eb02 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.801129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9935db9afd624fbb9c5a1da9273d60ed', 'request_id': 'req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:02.813235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe079c1-3346-4428-8a79-c4e04940eb02 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.815239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9935db9afd624fbb9c5a1da9273d60ed', 'request_id': 'req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:02.826618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9935db9afd624fbb9c5a1da9273d60ed', 'request_id': 'req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:02.847404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] The token cache is being invalidate because project 9935db9afd624fbb9c5a1da9273d60ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:02.847703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:02.848397 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7b937e47-f0cc-44a2-974e-3b83c0ce0e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:02.849982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1041/2097] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:02 2022] DELETE /identity/v3/projects/9935db9afd624fbb9c5a1da9273d60ed => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:02.851243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c19e-000b-48ff-8df0-afadd9865a64 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.851551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c19e-000b-48ff-8df0-afadd9865a64 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.851799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0c19e-000b-48ff-8df0-afadd9865a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.894709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.895472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.895823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.896225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.897249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.899280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:02.900715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-85c0c19e-000b-48ff-8df0-afadd9865a64 None None] MFA Rules not processed for user `a28a0c41c43246afa22643b472e70cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:02.902238 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-85c0c19e-000b-48ff-8df0-afadd9865a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:02.913979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1057/2098] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:02.915007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:02.920651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:02.934885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8453f2-6692-404d-8f07-f11db15df2cd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:02.941211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:02.942561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:02.943756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:02.944248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] PATH_INFO: `/v3/users/01fa5689c515459686aafa27872d1c7a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:02.946465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01fa5689c515459686aafa27872d1c7a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:02.951380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:03.004157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1042/2099] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:02 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:03.006074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0c19e-000b-48ff-8df0-afadd9865a64 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:03.010919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-85c0c19e-000b-48ff-8df0-afadd9865a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:03.032351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0c19e-000b-48ff-8df0-afadd9865a64 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:03.040118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01fa5689c515459686aafa27872d1c7a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.043740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:03.044965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:03.044965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:03.045294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:03.045879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:03.047701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:03.061718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1043/2100] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:03.061718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:03.067452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:03.070456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01fa5689c515459686aafa27872d1c7a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.071707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01fa5689c515459686aafa27872d1c7a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.083359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01fa5689c515459686aafa27872d1c7a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.094519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01fa5689c515459686aafa27872d1c7a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.104189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1058/2101] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:02 2022] DELETE /identity/v3/users/01fa5689c515459686aafa27872d1c7a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:03.105113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:03.107070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:03.107824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc2ce8-92e0-4987-8c8b-943101a419f2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:03.203244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:03.224803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:03.225658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:03.226380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:03.226670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] PATH_INFO: `/v3/users/57d383978b7248298cc7e962b7b04615` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:03.228917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d383978b7248298cc7e962b7b04615)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:03.233207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:03.252499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda1851-baf7-4bc9-bdb0-8072c4255e59 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:03.260220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:03.261000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:03.261353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:03.261735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:03.263599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:03.265096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:03.289720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1059/2102] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:03.289720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:03.292512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:03.298371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:03.330717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d383978b7248298cc7e962b7b04615'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.339014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d383978b7248298cc7e962b7b04615'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.341589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d383978b7248298cc7e962b7b04615'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.358378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d383978b7248298cc7e962b7b04615'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.379664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d383978b7248298cc7e962b7b04615'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:03.398481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1044/2103] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:03 2022] DELETE /identity/v3/users/57d383978b7248298cc7e962b7b04615 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:03.400158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:03.401759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:03.435852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0c62-8785-4510-a5b2-e54478b3634e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:03.567605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94f475-7e1e-4f30-bbc2-aade0e03b4ad admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:03.580286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:03.582615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:03.584004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:03.584677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:03.601215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:03.605926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:03.624949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:03.628112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:03.628614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:03.629339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:03.633655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:03.664137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1045/2104] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:03.666410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:03.669890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:03.707771 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c92729-8dfa-44a4-b6bd-074477fc6848 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:03.726380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db34569-9415-466c-8c55-f01535752a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:03.728382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34569-9415-466c-8c55-f01535752a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:03.728382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34569-9415-466c-8c55-f01535752a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:03.729440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db34569-9415-466c-8c55-f01535752a6f admin admin] PATH_INFO: `/v3/projects/4dfbeb0b9e1142359133d751698680c2/users/dd779702d38b41d4902fdcf891fdc052/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:03.763903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db34569-9415-466c-8c55-f01535752a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dfbeb0b9e1142359133d751698680c2, user_id=dd779702d38b41d4902fdcf891fdc052, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:03.769834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db34569-9415-466c-8c55-f01535752a6f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:03.770153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:03.771757 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:03.793892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1060/2105] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:03 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:03.797692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:03.807183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:03.827015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1046/2106] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:03 2022] PUT /identity/v3/projects/4dfbeb0b9e1142359133d751698680c2/users/dd779702d38b41d4902fdcf891fdc052/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:03.827015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db34569-9415-466c-8c55-f01535752a6f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:03.831833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03307baf-faee-41cb-a81e-3e891476c279 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:03.874577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:03.925131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:03.929314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:03.929314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:03.929314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] PATH_INFO: `/v3/projects/35d3c249756a4c969492e61ea3f6029c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:03.932033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] RBAC: Authorizing `identity:delete_project(project_id=35d3c249756a4c969492e61ea3f6029c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:03.934870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:04.012367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35d3c249756a4c969492e61ea3f6029c', 'request_id': 'req-2bd1c731-4afe-400c-b8fa-df966e6fcfee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.018115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db34569-9415-466c-8c55-f01535752a6f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:04.026802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35d3c249756a4c969492e61ea3f6029c', 'request_id': 'req-2bd1c731-4afe-400c-b8fa-df966e6fcfee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.042150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35d3c249756a4c969492e61ea3f6029c', 'request_id': 'req-2bd1c731-4afe-400c-b8fa-df966e6fcfee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.063508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] The token cache is being invalidate because project 35d3c249756a4c969492e61ea3f6029c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:04.064150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.065284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bd1c731-4afe-400c-b8fa-df966e6fcfee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.072610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1061/2107] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:03 2022] DELETE /identity/v3/projects/35d3c249756a4c969492e61ea3f6029c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:04.072610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce070ee-ed9e-4f40-b36f-da40f4913fd6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.072610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce070ee-ed9e-4f40-b36f-da40f4913fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.073886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce070ee-ed9e-4f40-b36f-da40f4913fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:04.087849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:04.125202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7ce070ee-ed9e-4f40-b36f-da40f4913fd6 None None] MFA Rules not processed for user `dd779702d38b41d4902fdcf891fdc052`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:04.127031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7ce070ee-ed9e-4f40-b36f-da40f4913fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.146770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:04.147707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.148038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.148574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] PATH_INFO: `/v3/projects/8a19baea189b401b91c3b08fe37e9abd/users/f7c0241c0ad945c8b01b9f0247624ab3/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:04.172134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a19baea189b401b91c3b08fe37e9abd, user_id=f7c0241c0ad945c8b01b9f0247624ab3, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:04.175210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:04.191880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1047/2108] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:03 2022] PUT /identity/v3/projects/8a19baea189b401b91c3b08fe37e9abd/users/f7c0241c0ad945c8b01b9f0247624ab3/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:04.193013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:04.194762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.231006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1062/2109] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:04.233674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce070ee-ed9e-4f40-b36f-da40f4913fd6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:04.233674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7ce070ee-ed9e-4f40-b36f-da40f4913fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.251587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce070ee-ed9e-4f40-b36f-da40f4913fd6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:04.285240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68028b5-e6da-418a-9744-b010630eb36b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:04.290938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:04.291618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.291903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.292191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] PATH_INFO: `/v3/projects/117dc432704c4c81b7c91c8c4b57e785` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:04.293438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=117dc432704c4c81b7c91c8c4b57e785)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:04.295508 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:04.311640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:04.330890 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:04.337211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.337211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.337211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] PATH_INFO: `/v3/users/076943a35907479f8759c423ed4871d9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:04.337211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=076943a35907479f8759c423ed4871d9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:04.337211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:04.365304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '117dc432704c4c81b7c91c8c4b57e785', 'request_id': 'req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.376140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '117dc432704c4c81b7c91c8c4b57e785', 'request_id': 'req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.385759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '117dc432704c4c81b7c91c8c4b57e785', 'request_id': 'req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.400960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '076943a35907479f8759c423ed4871d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.406643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] The token cache is being invalidate because project 117dc432704c4c81b7c91c8c4b57e785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:04.407019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.407934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.409763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1048/2110] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:04 2022] DELETE /identity/v3/projects/117dc432704c4c81b7c91c8c4b57e785 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:04.410899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:04.412294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.418769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '076943a35907479f8759c423ed4871d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.419905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '076943a35907479f8759c423ed4871d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.431237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '076943a35907479f8759c423ed4871d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.445282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-80425667-3b7e-44d2-976f-ec6e1a294183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '076943a35907479f8759c423ed4871d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:04.453536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1063/2111] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:04 2022] DELETE /identity/v3/users/076943a35907479f8759c423ed4871d9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:04.455025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467eb03f-8cf2-49b9-bc53-c0ef55da92ba None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.455363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467eb03f-8cf2-49b9-bc53-c0ef55da92ba None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.455746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467eb03f-8cf2-49b9-bc53-c0ef55da92ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:04.477962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:04.497605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-467eb03f-8cf2-49b9-bc53-c0ef55da92ba None None] MFA Rules not processed for user `f7c0241c0ad945c8b01b9f0247624ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:04.499138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-467eb03f-8cf2-49b9-bc53-c0ef55da92ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.502517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a46bbdd-5a5c-4514-81fe-74f8e3e11a0d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:04.592946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:04.594173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.594744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.595203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:04.597911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.616353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1064/2112] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:04.616353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467eb03f-8cf2-49b9-bc53-c0ef55da92ba None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:04.616353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-467eb03f-8cf2-49b9-bc53-c0ef55da92ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.745055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:04.749054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:04.751410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467eb03f-8cf2-49b9-bc53-c0ef55da92ba None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:04.772737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:04.773810 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.774054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.774397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:04.775145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:04.778591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:04.786225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:04.805976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1049/2113] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:04 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:04.816797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:04.819800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.842769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1065/2114] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:04 2022] POST /identity/v3/projects => generated 387 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:04.843783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:04.847708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccc5a2-e2b1-4d93-b64e-100d97e8c3b1 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:04.849181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:04.988620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e4969d-fa4d-4ec9-bc47-578b32b9332d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:04.991401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:04.992924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.993210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.993886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] PATH_INFO: `/v3/users/fbb62dc3060a4e479bb2112cc946120b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:04.995987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb62dc3060a4e479bb2112cc946120b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:04.997948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:04.998869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:04.999050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:04.999484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:04.999978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.001382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.005558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.063830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1066/2115] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:04 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:05.065235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:05.070743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:05.072655 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb62dc3060a4e479bb2112cc946120b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.073998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb62dc3060a4e479bb2112cc946120b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.075485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb62dc3060a4e479bb2112cc946120b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.085989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:05.088601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb62dc3060a4e479bb2112cc946120b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.108243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb62dc3060a4e479bb2112cc946120b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.134900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1050/2116] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:04 2022] DELETE /identity/v3/users/fbb62dc3060a4e479bb2112cc946120b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:05.136686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:05.140147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:05.219915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b1c1-e0d7-4e4c-8582-9480d9d3269e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:05.314497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb700d0-0609-4abb-a836-11571a4255dd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:05.347023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:05.348122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.348784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.349282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.350740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.362034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.407715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:05.409178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.409966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.410184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1051/2117] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:05 2022] POST /identity/v3/users => generated 300 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:05.410829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] PATH_INFO: `/v3/projects/443b0cf1604e4b74ab4f4db3ddb37bd8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.412151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:05.414645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=443b0cf1604e4b74ab4f4db3ddb37bd8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.416823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:05.420206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.447035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff5d2b-f025-449e-a295-2c6ef739bbab admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:05.459469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53c15a-08e7-49b9-8f5d-51ef65682b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:05.460703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c15a-08e7-49b9-8f5d-51ef65682b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.461216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c15a-08e7-49b9-8f5d-51ef65682b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.462413 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53c15a-08e7-49b9-8f5d-51ef65682b24 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.463679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53c15a-08e7-49b9-8f5d-51ef65682b24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.468829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53c15a-08e7-49b9-8f5d-51ef65682b24 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.475570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:05.521303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1052/2118] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:05 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:05.525174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53c15a-08e7-49b9-8f5d-51ef65682b24 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:05.541941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53c15a-08e7-49b9-8f5d-51ef65682b24 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:05.548639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '443b0cf1604e4b74ab4f4db3ddb37bd8', 'request_id': 'req-5ebcd8c1-96d2-4664-bc0e-da8555599021'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.564370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '443b0cf1604e4b74ab4f4db3ddb37bd8', 'request_id': 'req-5ebcd8c1-96d2-4664-bc0e-da8555599021'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.581136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '443b0cf1604e4b74ab4f4db3ddb37bd8', 'request_id': 'req-5ebcd8c1-96d2-4664-bc0e-da8555599021'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.610169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] The token cache is being invalidate because project 443b0cf1604e4b74ab4f4db3ddb37bd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:05.610946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.612685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:05.617677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1067/2119] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:05 2022] DELETE /identity/v3/projects/443b0cf1604e4b74ab4f4db3ddb37bd8 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:05.619326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:05.621733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:05.636701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00412f22-7a56-4feb-b702-bbe175c9e155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:05.638013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412f22-7a56-4feb-b702-bbe175c9e155 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.638369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412f22-7a56-4feb-b702-bbe175c9e155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.638710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00412f22-7a56-4feb-b702-bbe175c9e155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.639518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00412f22-7a56-4feb-b702-bbe175c9e155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.642959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00412f22-7a56-4feb-b702-bbe175c9e155 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.664958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1053/2120] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:05.664958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2953b61-7f6a-4841-a220-3cd1d68d9fe6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.664958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2953b61-7f6a-4841-a220-3cd1d68d9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.664958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2953b61-7f6a-4841-a220-3cd1d68d9fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.692482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c2953b61-7f6a-4841-a220-3cd1d68d9fe6 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:05.694288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c2953b61-7f6a-4841-a220-3cd1d68d9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:05.707066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1054/2121] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:05.708511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2953b61-7f6a-4841-a220-3cd1d68d9fe6 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:05.710169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c2953b61-7f6a-4841-a220-3cd1d68d9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:05.735851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2953b61-7f6a-4841-a220-3cd1d68d9fe6 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:05.747152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:05.748887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.749878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.750405 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] PATH_INFO: `/v3/projects/4536300a8610460c8704926cdc49b230/users/ac1fea4826274e35940c3ad7c9552158/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.757772 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcd8c1-96d2-4664-bc0e-da8555599021 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:05.764302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c7c70-4d68-4ee4-8f15-0976116d3fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:05.765239 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c7c70-4d68-4ee4-8f15-0976116d3fec admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.765361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c7c70-4d68-4ee4-8f15-0976116d3fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.765599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c7c70-4d68-4ee4-8f15-0976116d3fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.766279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c7c70-4d68-4ee4-8f15-0976116d3fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.768461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c7c70-4d68-4ee4-8f15-0976116d3fec admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.785301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1068/2122] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:05.786857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f5fd-2dc5-4c39-8f2e-6d1657e5bb55 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.787153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f5fd-2dc5-4c39-8f2e-6d1657e5bb55 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.787495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224f5fd-2dc5-4c39-8f2e-6d1657e5bb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.792623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4536300a8610460c8704926cdc49b230, user_id=ac1fea4826274e35940c3ad7c9552158, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.797567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.816843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d224f5fd-2dc5-4c39-8f2e-6d1657e5bb55 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:05.819209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d224f5fd-2dc5-4c39-8f2e-6d1657e5bb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:05.827377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1055/2123] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:05 2022] PUT /identity/v3/projects/4536300a8610460c8704926cdc49b230/users/ac1fea4826274e35940c3ad7c9552158/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:05.827377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:05.829840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:05.844928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=3,keystone:INSERT=8 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:05.908531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1069/2124] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:05.909809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f5fd-2dc5-4c39-8f2e-6d1657e5bb55 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:05.921623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224f5fd-2dc5-4c39-8f2e-6d1657e5bb55 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:05.926714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482df8b-392b-4f65-9605-9eb9aa677262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:05.927788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482df8b-392b-4f65-9605-9eb9aa677262 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.927974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482df8b-392b-4f65-9605-9eb9aa677262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.928375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482df8b-392b-4f65-9605-9eb9aa677262 admin admin] PATH_INFO: `/v3/projects/0bc2032a49a04794aef45bc0c45a1318/users/ec488cb7d7e84742947334af92c7900d/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.940196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bccac-640a-43e2-bfef-a6a825560821 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:05.946157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482df8b-392b-4f65-9605-9eb9aa677262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc2032a49a04794aef45bc0c45a1318, user_id=ec488cb7d7e84742947334af92c7900d, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.950276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482df8b-392b-4f65-9605-9eb9aa677262 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.951586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:05.952814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:05.953666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:05.954576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] PATH_INFO: `/v3/projects/7c5d5a21bd8d4596bde9f0b6dd426e7f` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:05.958637 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c5d5a21bd8d4596bde9f0b6dd426e7f)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:05.964227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:05.971475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1070/2125] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:05 2022] PUT /identity/v3/projects/0bc2032a49a04794aef45bc0c45a1318/users/ec488cb7d7e84742947334af92c7900d/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:05.972753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482df8b-392b-4f65-9605-9eb9aa677262 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.025088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:06.057534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c5d5a21bd8d4596bde9f0b6dd426e7f', 'request_id': 'req-704d58c8-c5a3-4b08-94d5-890f9a31e408'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:06.068079 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c5d5a21bd8d4596bde9f0b6dd426e7f', 'request_id': 'req-704d58c8-c5a3-4b08-94d5-890f9a31e408'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:06.076070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c5d5a21bd8d4596bde9f0b6dd426e7f', 'request_id': 'req-704d58c8-c5a3-4b08-94d5-890f9a31e408'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:06.091918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] The token cache is being invalidate because project 7c5d5a21bd8d4596bde9f0b6dd426e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:06.092352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:06.093337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:06.101580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1056/2126] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:05 2022] DELETE /identity/v3/projects/7c5d5a21bd8d4596bde9f0b6dd426e7f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:06.103102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.109055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:06.120249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482df8b-392b-4f65-9605-9eb9aa677262 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.251107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d58c8-c5a3-4b08-94d5-890f9a31e408 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.258533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.259543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.259883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.260237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.260994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:06.261364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.263211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.265889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e388d25-796a-40f1-84f8-09fdced883de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.268248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388d25-796a-40f1-84f8-09fdced883de admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.269039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388d25-796a-40f1-84f8-09fdced883de admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.269489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388d25-796a-40f1-84f8-09fdced883de admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.270332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e388d25-796a-40f1-84f8-09fdced883de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.272444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e388d25-796a-40f1-84f8-09fdced883de admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.285741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1057/2127] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:06.286993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.289516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1071/2128] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:06.289758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:06.295925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e388d25-796a-40f1-84f8-09fdced883de admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.307559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e388d25-796a-40f1-84f8-09fdced883de admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.310419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b581a-03ec-4791-b4b3-0b23056b34aa admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.314313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42061a-bcdb-4a58-8113-4c1ccd4a0331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.315092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42061a-bcdb-4a58-8113-4c1ccd4a0331 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.315443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42061a-bcdb-4a58-8113-4c1ccd4a0331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.315796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42061a-bcdb-4a58-8113-4c1ccd4a0331 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.316063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.316624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42061a-bcdb-4a58-8113-4c1ccd4a0331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.316792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.317016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.317235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.317949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.318930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42061a-bcdb-4a58-8113-4c1ccd4a0331 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.319593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.346781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1058/2129] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:06.347945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.351170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:06.362715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1072/2130] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:06 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:06.365371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42061a-bcdb-4a58-8113-4c1ccd4a0331 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.382532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79cdc3-ede4-42cc-abe7-d3b7e731b678 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.408918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:06.453503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42061a-bcdb-4a58-8113-4c1ccd4a0331 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.462625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684917b-fd7d-49a7-a192-0de0066a3cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.463313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684917b-fd7d-49a7-a192-0de0066a3cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.463594 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684917b-fd7d-49a7-a192-0de0066a3cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.463878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684917b-fd7d-49a7-a192-0de0066a3cc1 admin admin] PATH_INFO: `/v3/projects/0bc2032a49a04794aef45bc0c45a1318/users/ec488cb7d7e84742947334af92c7900d/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.467223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684917b-fd7d-49a7-a192-0de0066a3cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc2032a49a04794aef45bc0c45a1318, user_id=ec488cb7d7e84742947334af92c7900d, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.469358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd05b2-0963-4eb7-974a-5cbbe1b4352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.470200 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd05b2-0963-4eb7-974a-5cbbe1b4352b admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.470486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd05b2-0963-4eb7-974a-5cbbe1b4352b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.470767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd05b2-0963-4eb7-974a-5cbbe1b4352b admin admin] PATH_INFO: `/v3/projects/4536300a8610460c8704926cdc49b230/users/ac1fea4826274e35940c3ad7c9552158/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.472845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd05b2-0963-4eb7-974a-5cbbe1b4352b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4536300a8610460c8704926cdc49b230, user_id=ac1fea4826274e35940c3ad7c9552158, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.473611 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684917b-fd7d-49a7-a192-0de0066a3cc1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.475580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd05b2-0963-4eb7-974a-5cbbe1b4352b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.488215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1059/2131] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:06 2022] PUT /identity/v3/projects/4536300a8610460c8704926cdc49b230/users/ac1fea4826274e35940c3ad7c9552158/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:06.489062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd05b2-0963-4eb7-974a-5cbbe1b4352b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.506049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1073/2132] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:06 2022] PUT /identity/v3/projects/0bc2032a49a04794aef45bc0c45a1318/users/ec488cb7d7e84742947334af92c7900d/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:06.506049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e2b2-fb27-4bf9-a66f-87c39df07fb2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.506049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e2b2-fb27-4bf9-a66f-87c39df07fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.506538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e2b2-fb27-4bf9-a66f-87c39df07fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.554881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6c70e2b2-fb27-4bf9-a66f-87c39df07fb2 None None] MFA Rules not processed for user `ac1fea4826274e35940c3ad7c9552158`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:06.557490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6c70e2b2-fb27-4bf9-a66f-87c39df07fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:06.611628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:06.612036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd05b2-0963-4eb7-974a-5cbbe1b4352b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.694256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1074/2133] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:06.695694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ae88b-2f20-4382-a144-10638da13edd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.695992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ae88b-2f20-4382-a144-10638da13edd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.696278 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ae88b-2f20-4382-a144-10638da13edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.715452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.716312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.716770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.716770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.717594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.719465 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.754875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1060/2134] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:06 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:06.755155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-076ae88b-2f20-4382-a144-10638da13edd None None] MFA Rules not processed for user `ec488cb7d7e84742947334af92c7900d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:06.755325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.755421 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-076ae88b-2f20-4382-a144-10638da13edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:06.755512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:06.820841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1311f4-ebf0-463f-8fb8-14a08166c2bf admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.825440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.826267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.826552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.826832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.835595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:06.839002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.840811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.842292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1075/2135] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:06.842881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ae88b-2f20-4382-a144-10638da13edd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.853030 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1061/2136] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:06.857929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ae88b-2f20-4382-a144-10638da13edd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.860700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.862387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:06.862987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce89c0-4d6b-4465-8156-bfae3a270db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.863604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce89c0-4d6b-4465-8156-bfae3a270db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.863888 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce89c0-4d6b-4465-8156-bfae3a270db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.864162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce89c0-4d6b-4465-8156-bfae3a270db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.864817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce89c0-4d6b-4465-8156-bfae3a270db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.866455 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce89c0-4d6b-4465-8156-bfae3a270db2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.876138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d62b3a9-87dc-4e30-9605-bb15ac062f7f service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.881281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dbb96-5753-4f0c-99fa-8265848947bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.881949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dbb96-5753-4f0c-99fa-8265848947bb service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.882286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dbb96-5753-4f0c-99fa-8265848947bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.882504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dbb96-5753-4f0c-99fa-8265848947bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.888004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1076/2137] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:06.889636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce89c0-4d6b-4465-8156-bfae3a270db2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.900637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce89c0-4d6b-4465-8156-bfae3a270db2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.901327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09dbb96-5753-4f0c-99fa-8265848947bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.903445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09dbb96-5753-4f0c-99fa-8265848947bb service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.905980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87557aa-ed25-40e6-99f2-f6a2a8af8188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.906663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87557aa-ed25-40e6-99f2-f6a2a8af8188 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.906967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87557aa-ed25-40e6-99f2-f6a2a8af8188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.907254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87557aa-ed25-40e6-99f2-f6a2a8af8188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.907907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87557aa-ed25-40e6-99f2-f6a2a8af8188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.909541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87557aa-ed25-40e6-99f2-f6a2a8af8188 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.916648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1062/2138] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:06.926330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1077/2139] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:06.930927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dbb96-5753-4f0c-99fa-8265848947bb service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.948395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dbb96-5753-4f0c-99fa-8265848947bb service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.949537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87557aa-ed25-40e6-99f2-f6a2a8af8188 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:06.958248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55e40d-b523-4142-a9d1-409bee8cd6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.958937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e40d-b523-4142-a9d1-409bee8cd6b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.959217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e40d-b523-4142-a9d1-409bee8cd6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.959481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e40d-b523-4142-a9d1-409bee8cd6b4 admin admin] PATH_INFO: `/v3/projects/9d0fc25bdc2d48349eb918c095ae2bbf/users/9e5811e50e1e433bb888f0455d2ddca0/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.962215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87557aa-ed25-40e6-99f2-f6a2a8af8188 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:06.971552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:06.972212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:06.972832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:06.973543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] PATH_INFO: `/v3/users/f7c0241c0ad945c8b01b9f0247624ab3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:06.975127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c0241c0ad945c8b01b9f0247624ab3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.977193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.977963 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55e40d-b523-4142-a9d1-409bee8cd6b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0fc25bdc2d48349eb918c095ae2bbf, user_id=9e5811e50e1e433bb888f0455d2ddca0, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:06.981944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55e40d-b523-4142-a9d1-409bee8cd6b4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:06.998368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1063/2140] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:06 2022] PUT /identity/v3/projects/9d0fc25bdc2d48349eb918c095ae2bbf/users/9e5811e50e1e433bb888f0455d2ddca0/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:07.008435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd076b0-25b3-494a-b243-86fd3b78daf1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.008779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd076b0-25b3-494a-b243-86fd3b78daf1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.008993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd076b0-25b3-494a-b243-86fd3b78daf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.045279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3bd076b0-25b3-494a-b243-86fd3b78daf1 None None] MFA Rules not processed for user `9e5811e50e1e433bb888f0455d2ddca0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:07.046749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3bd076b0-25b3-494a-b243-86fd3b78daf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.052473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c0241c0ad945c8b01b9f0247624ab3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.066541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c0241c0ad945c8b01b9f0247624ab3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.067396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c0241c0ad945c8b01b9f0247624ab3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.083033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c0241c0ad945c8b01b9f0247624ab3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.095505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c0241c0ad945c8b01b9f0247624ab3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.097949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:07.103853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1078/2141] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:06 2022] DELETE /identity/v3/users/f7c0241c0ad945c8b01b9f0247624ab3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:07.104977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:07.109379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.144191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1064/2142] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:07.145978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fc4c0-130c-4d86-b990-9886da826952 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.146474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fc4c0-130c-4d86-b990-9886da826952 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.146952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fc4c0-130c-4d86-b990-9886da826952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.174280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5b1fc4c0-130c-4d86-b990-9886da826952 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:07.175998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5b1fc4c0-130c-4d86-b990-9886da826952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.228489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7f6a0-1b7c-4cb1-8948-f3a31f708554 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:07.235172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:07.235172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.235488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.235857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] PATH_INFO: `/v3/users/dd779702d38b41d4902fdcf891fdc052` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.237128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd779702d38b41d4902fdcf891fdc052)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:07.239658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:07.270521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1065/2143] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:07.272335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fc4c0-130c-4d86-b990-9886da826952 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:07.275412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5b1fc4c0-130c-4d86-b990-9886da826952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.308346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd779702d38b41d4902fdcf891fdc052'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.329181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd779702d38b41d4902fdcf891fdc052'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.330694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd779702d38b41d4902fdcf891fdc052'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.344631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd779702d38b41d4902fdcf891fdc052'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.358945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd779702d38b41d4902fdcf891fdc052'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.374984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1079/2144] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:07 2022] DELETE /identity/v3/users/dd779702d38b41d4902fdcf891fdc052 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:07.374984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:07.375959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:07.376110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.384424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fc4c0-130c-4d86-b990-9886da826952 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:07.455108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e147248-2fed-4910-8604-488005914000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:07.455737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e147248-2fed-4910-8604-488005914000 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.456131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e147248-2fed-4910-8604-488005914000 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.456283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e147248-2fed-4910-8604-488005914000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.458540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2e147248-2fed-4910-8604-488005914000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.465753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d74ab-6032-4480-9f85-ec8487b404dc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:07.470012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:07.470525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.470786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.472995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.474000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.510159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:07.541642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e147248-2fed-4910-8604-488005914000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:07.544386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e147248-2fed-4910-8604-488005914000 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:07.555307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:07.555577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1066/2145] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:07.556929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eb000-d244-4509-a4e1-4b1f1918f05e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.557222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eb000-d244-4509-a4e1-4b1f1918f05e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.557530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8eb000-d244-4509-a4e1-4b1f1918f05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.560122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:07.583647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1080/2146] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:07.585112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:07.587002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0d8eb000-d244-4509-a4e1-4b1f1918f05e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:07.587351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.588969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0d8eb000-d244-4509-a4e1-4b1f1918f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.606169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1067/2147] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:07.615000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2420fa6-b736-400f-8c7c-0b3d2d005277 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:07.623044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eb000-d244-4509-a4e1-4b1f1918f05e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:07.624211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:07.625339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0d8eb000-d244-4509-a4e1-4b1f1918f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.625529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.625914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.626365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] PATH_INFO: `/v3/users/9e5811e50e1e433bb888f0455d2ddca0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.628803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5811e50e1e433bb888f0455d2ddca0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:07.632404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:07.647663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8eb000-d244-4509-a4e1-4b1f1918f05e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:07.654257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ea53d-ae7b-4cfb-88cc-d86c1b9d2f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:07.655025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea53d-ae7b-4cfb-88cc-d86c1b9d2f85 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.655471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea53d-ae7b-4cfb-88cc-d86c1b9d2f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.655764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea53d-ae7b-4cfb-88cc-d86c1b9d2f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.669147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ea53d-ae7b-4cfb-88cc-d86c1b9d2f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:07.670843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ea53d-ae7b-4cfb-88cc-d86c1b9d2f85 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:07.686230 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1068/2148] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:07.722340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5811e50e1e433bb888f0455d2ddca0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.764898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5811e50e1e433bb888f0455d2ddca0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.764898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5811e50e1e433bb888f0455d2ddca0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.787097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5811e50e1e433bb888f0455d2ddca0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.805558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5811e50e1e433bb888f0455d2ddca0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:07.831113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1081/2149] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:07 2022] DELETE /identity/v3/users/9e5811e50e1e433bb888f0455d2ddca0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:07.841723 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92952bd-ad16-4400-86e7-ec4788d5e589 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:07.842216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92952bd-ad16-4400-86e7-ec4788d5e589 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:07.842672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92952bd-ad16-4400-86e7-ec4788d5e589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:07.881117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b92952bd-ad16-4400-86e7-ec4788d5e589 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:07.882699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b92952bd-ad16-4400-86e7-ec4788d5e589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:07.993018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:07.995512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:08.021850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1069/2150] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:08.046268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92952bd-ad16-4400-86e7-ec4788d5e589 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:08.048992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b92952bd-ad16-4400-86e7-ec4788d5e589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:08.145654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e8a7c-2101-4369-9c2e-4e50d6c41cce admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:08.151179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:08.151916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:08.152800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:08.153933 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] PATH_INFO: `/v3/projects/8a19baea189b401b91c3b08fe37e9abd` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:08.155476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a19baea189b401b91c3b08fe37e9abd)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:08.157612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:08.174114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:08.206692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92952bd-ad16-4400-86e7-ec4788d5e589 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:08.218859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:08.219921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:08.220393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:08.221503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:08.230274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:08.251611 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a19baea189b401b91c3b08fe37e9abd', 'request_id': 'req-7809cf4a-8c5b-46e1-b7b6-54decde38f53'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.252524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:08.255536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:08.275076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a19baea189b401b91c3b08fe37e9abd', 'request_id': 'req-7809cf4a-8c5b-46e1-b7b6-54decde38f53'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.279092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1070/2151] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:08.281418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:08.286297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:08.295497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a19baea189b401b91c3b08fe37e9abd', 'request_id': 'req-7809cf4a-8c5b-46e1-b7b6-54decde38f53'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.314366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d5a6f-575b-4950-a610-b6d4ee619aa6 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:08.324271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] The token cache is being invalidate because project 8a19baea189b401b91c3b08fe37e9abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:08.325386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.327815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.337593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1082/2152] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:07 2022] DELETE /identity/v3/projects/8a19baea189b401b91c3b08fe37e9abd => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:08.340046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:08.354362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:08.437569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:08.439234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:08.439907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:08.442703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] PATH_INFO: `/v3/projects/4dfbeb0b9e1142359133d751698680c2` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:08.445563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dfbeb0b9e1142359133d751698680c2)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:08.452246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:08.516645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809cf4a-8c5b-46e1-b7b6-54decde38f53 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:08.531736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:08.533045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:08.533867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:08.534453 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] PATH_INFO: `/v3/users/ec488cb7d7e84742947334af92c7900d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:08.538449 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec488cb7d7e84742947334af92c7900d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:08.546985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:08.598152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dfbeb0b9e1142359133d751698680c2', 'request_id': 'req-7601cabb-af49-4c09-8784-a4c6ce6e2016'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.609371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dfbeb0b9e1142359133d751698680c2', 'request_id': 'req-7601cabb-af49-4c09-8784-a4c6ce6e2016'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.619888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dfbeb0b9e1142359133d751698680c2', 'request_id': 'req-7601cabb-af49-4c09-8784-a4c6ce6e2016'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.643013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] The token cache is being invalidate because project 4dfbeb0b9e1142359133d751698680c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:08.643377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.644296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.646337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1071/2153] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:08 2022] DELETE /identity/v3/projects/4dfbeb0b9e1142359133d751698680c2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:08.650313 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:08.652105 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:08.690065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec488cb7d7e84742947334af92c7900d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.719326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec488cb7d7e84742947334af92c7900d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.720264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec488cb7d7e84742947334af92c7900d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.730396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec488cb7d7e84742947334af92c7900d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.743218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec488cb7d7e84742947334af92c7900d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.759499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1083/2154] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:08 2022] DELETE /identity/v3/users/ec488cb7d7e84742947334af92c7900d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:08.771323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7601cabb-af49-4c09-8784-a4c6ce6e2016 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:08.811518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:08.891009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:08.891722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:08.892009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:08.892340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] PATH_INFO: `/v3/users/ac1fea4826274e35940c3ad7c9552158` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:08.893628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1fea4826274e35940c3ad7c9552158)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:08.895638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:08.958230 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1fea4826274e35940c3ad7c9552158'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.959082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1fea4826274e35940c3ad7c9552158'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.960038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1fea4826274e35940c3ad7c9552158'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.977899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1fea4826274e35940c3ad7c9552158'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:08.999765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1fea4826274e35940c3ad7c9552158'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.020643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1072/2155] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:08 2022] DELETE /identity/v3/users/ac1fea4826274e35940c3ad7c9552158 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:09.036238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:09.038786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.061084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ab36e-cdcc-4297-8ce0-600a028454be admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:09.130620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:09.150647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:09.151412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.151706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.152027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] PATH_INFO: `/v3/projects/9d0fc25bdc2d48349eb918c095ae2bbf` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.155029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0fc25bdc2d48349eb918c095ae2bbf)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:09.157907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:09.226856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0fc25bdc2d48349eb918c095ae2bbf', 'request_id': 'req-503d5757-5040-4203-b33d-f409ddf58c10'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.235405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0fc25bdc2d48349eb918c095ae2bbf', 'request_id': 'req-503d5757-5040-4203-b33d-f409ddf58c10'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.254036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0fc25bdc2d48349eb918c095ae2bbf', 'request_id': 'req-503d5757-5040-4203-b33d-f409ddf58c10'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.270737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] The token cache is being invalidate because project 9d0fc25bdc2d48349eb918c095ae2bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:09.271157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.271997 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-503d5757-5040-4203-b33d-f409ddf58c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.277488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1084/2156] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:09 2022] DELETE /identity/v3/projects/9d0fc25bdc2d48349eb918c095ae2bbf => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:09.277702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:09.285690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.286337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ef38-4545-4602-a469-d5347f70cc84 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.286337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ef38-4545-4602-a469-d5347f70cc84 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.286337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8ef38-4545-4602-a469-d5347f70cc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.300225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d9bca-c579-481b-b05d-326208da3b38 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:09.311045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f2f8ef38-4545-4602-a469-d5347f70cc84 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:09.312439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f2f8ef38-4545-4602-a469-d5347f70cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.387475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:09.387949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.388233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.388684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] PATH_INFO: `/v3/projects/0bc2032a49a04794aef45bc0c45a1318` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.389952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bc2032a49a04794aef45bc0c45a1318)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:09.392097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:09.419984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1085/2157] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:09.430383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad0edc-958a-41b0-8c54-0c0e914ebd4b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.430859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad0edc-958a-41b0-8c54-0c0e914ebd4b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.431395 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad0edc-958a-41b0-8c54-0c0e914ebd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.451941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:09.456158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-28ad0edc-958a-41b0-8c54-0c0e914ebd4b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:09.457782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-28ad0edc-958a-41b0-8c54-0c0e914ebd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.471962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bc2032a49a04794aef45bc0c45a1318', 'request_id': 'req-24299cc9-ce2d-4f49-b766-78e673623233'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.484644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bc2032a49a04794aef45bc0c45a1318', 'request_id': 'req-24299cc9-ce2d-4f49-b766-78e673623233'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.493772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bc2032a49a04794aef45bc0c45a1318', 'request_id': 'req-24299cc9-ce2d-4f49-b766-78e673623233'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.522354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] The token cache is being invalidate because project 0bc2032a49a04794aef45bc0c45a1318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:09.522639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.523526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.526289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1073/2158] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:09 2022] DELETE /identity/v3/projects/0bc2032a49a04794aef45bc0c45a1318 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:09.531915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:09.533714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.581936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1086/2159] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:09.584424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99deeb-a29d-4d1c-8fd0-fcde61d1e1b7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.584919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99deeb-a29d-4d1c-8fd0-fcde61d1e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.585421 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99deeb-a29d-4d1c-8fd0-fcde61d1e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.615756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2b99deeb-a29d-4d1c-8fd0-fcde61d1e1b7 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:09.617808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2b99deeb-a29d-4d1c-8fd0-fcde61d1e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.631379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1087/2160] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:09.632884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99deeb-a29d-4d1c-8fd0-fcde61d1e1b7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:09.645597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24299cc9-ce2d-4f49-b766-78e673623233 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:09.650902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99deeb-a29d-4d1c-8fd0-fcde61d1e1b7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:09.651404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:09.652185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.653170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.653700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] PATH_INFO: `/v3/projects/4536300a8610460c8704926cdc49b230` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.655254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4536300a8610460c8704926cdc49b230)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:09.657371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:09.659496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c64ea-2557-4d2a-9ce1-b55065282db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:09.660560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c64ea-2557-4d2a-9ce1-b55065282db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.661070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c64ea-2557-4d2a-9ce1-b55065282db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.661618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c64ea-2557-4d2a-9ce1-b55065282db0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.662850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c64ea-2557-4d2a-9ce1-b55065282db0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:09.663388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c64ea-2557-4d2a-9ce1-b55065282db0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:09.666428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c64ea-2557-4d2a-9ce1-b55065282db0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:09.682690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:09.689957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1088/2161] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:09.696567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0eb1-95a0-4802-aef0-dd13426895c8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.697024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0eb1-95a0-4802-aef0-dd13426895c8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.697607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f0eb1-95a0-4802-aef0-dd13426895c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.733556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-db8f0eb1-95a0-4802-aef0-dd13426895c8 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:09.735307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-db8f0eb1-95a0-4802-aef0-dd13426895c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.737610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4536300a8610460c8704926cdc49b230', 'request_id': 'req-970ed79f-e8a4-4923-bfe6-1def11352118'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.750367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4536300a8610460c8704926cdc49b230', 'request_id': 'req-970ed79f-e8a4-4923-bfe6-1def11352118'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.763047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4536300a8610460c8704926cdc49b230', 'request_id': 'req-970ed79f-e8a4-4923-bfe6-1def11352118'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.788112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] The token cache is being invalidate because project 4536300a8610460c8704926cdc49b230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:09.788634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.789859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:09.794215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1074/2162] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:09 2022] DELETE /identity/v3/projects/4536300a8610460c8704926cdc49b230 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:09.795752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:09.798290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.847177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1089/2163] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:09.849561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2dc45-a3b5-487a-a53f-08048057f8e8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.850043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2dc45-a3b5-487a-a53f-08048057f8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.850336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2dc45-a3b5-487a-a53f-08048057f8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.898845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-91d2dc45-a3b5-487a-a53f-08048057f8e8 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:09.900205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-91d2dc45-a3b5-487a-a53f-08048057f8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:09.931441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1090/2164] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:09.932599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2dc45-a3b5-487a-a53f-08048057f8e8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:09.955831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2dc45-a3b5-487a-a53f-08048057f8e8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:09.962236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ed79f-e8a4-4923-bfe6-1def11352118 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:09.964545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd599a-4d8f-4a41-83a9-b918733e8ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:09.967106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd599a-4d8f-4a41-83a9-b918733e8ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.967791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd599a-4d8f-4a41-83a9-b918733e8ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.968886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd599a-4d8f-4a41-83a9-b918733e8ea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.969429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6917c7c-967d-420d-a510-f43ff53a4869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:09.970139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6917c7c-967d-420d-a510-f43ff53a4869 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:09.970426 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6917c7c-967d-420d-a510-f43ff53a4869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:09.970704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6917c7c-967d-420d-a510-f43ff53a4869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:09.970982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd599a-4d8f-4a41-83a9-b918733e8ea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:09.971359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6917c7c-967d-420d-a510-f43ff53a4869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:09.971676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd599a-4d8f-4a41-83a9-b918733e8ea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:09.974021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6917c7c-967d-420d-a510-f43ff53a4869 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:09.975477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd599a-4d8f-4a41-83a9-b918733e8ea1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:09.987025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:10.021889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1091/2165] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:10.023840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaba17-a910-4283-bea6-7f91f9e2430a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.024437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaba17-a910-4283-bea6-7f91f9e2430a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.025537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaba17-a910-4283-bea6-7f91f9e2430a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.036702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1075/2166] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:09 2022] POST /identity/v3/projects => generated 373 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:10.038383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6917c7c-967d-420d-a510-f43ff53a4869 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.054898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-68eaba17-a910-4283-bea6-7f91f9e2430a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:10.057091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-68eaba17-a910-4283-bea6-7f91f9e2430a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:10.157359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6917c7c-967d-420d-a510-f43ff53a4869 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.168730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d77d73-e043-4a00-92c5-b241218b5c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.169616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d77d73-e043-4a00-92c5-b241218b5c29 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.169912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d77d73-e043-4a00-92c5-b241218b5c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.170267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d77d73-e043-4a00-92c5-b241218b5c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.170940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d77d73-e043-4a00-92c5-b241218b5c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.172975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d77d73-e043-4a00-92c5-b241218b5c29 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.180867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1092/2167] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:10.180867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eaba17-a910-4283-bea6-7f91f9e2430a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.195710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eaba17-a910-4283-bea6-7f91f9e2430a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.201905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7d9a-ce51-4094-b61a-cf303667aad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.202714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7d9a-ce51-4094-b61a-cf303667aad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.203001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7d9a-ce51-4094-b61a-cf303667aad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.203296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7d9a-ce51-4094-b61a-cf303667aad5 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.203959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd7d9a-ce51-4094-b61a-cf303667aad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.206346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd7d9a-ce51-4094-b61a-cf303667aad5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.212786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1076/2168] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:10 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:10.214443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d77d73-e043-4a00-92c5-b241218b5c29 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.231276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1093/2169] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:10 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:10.235305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7d9a-ce51-4094-b61a-cf303667aad5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.247872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7d9a-ce51-4094-b61a-cf303667aad5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.329851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d77d73-e043-4a00-92c5-b241218b5c29 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.335160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.335808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.336100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.336418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.337040 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:10.337345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.339053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.344457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37832096-801b-4b2b-9eb6-dd396ac41c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.345633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37832096-801b-4b2b-9eb6-dd396ac41c17 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.346158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37832096-801b-4b2b-9eb6-dd396ac41c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.346884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37832096-801b-4b2b-9eb6-dd396ac41c17 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.348446 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37832096-801b-4b2b-9eb6-dd396ac41c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.352843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37832096-801b-4b2b-9eb6-dd396ac41c17 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.363129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1077/2170] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:10.363129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.375076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:10.378272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42499767-0f84-4d8c-bc90-d229195a6d37 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.384989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a08a-9bff-4df3-9a24-ea7c4d916688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.386237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a08a-9bff-4df3-9a24-ea7c4d916688 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.388571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a08a-9bff-4df3-9a24-ea7c4d916688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.389055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94a08a-9bff-4df3-9a24-ea7c4d916688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.390277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94a08a-9bff-4df3-9a24-ea7c4d916688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.393844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94a08a-9bff-4df3-9a24-ea7c4d916688 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.405963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1094/2171] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:10 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:10.407643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37832096-801b-4b2b-9eb6-dd396ac41c17 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.415202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1078/2172] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:10.418257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a08a-9bff-4df3-9a24-ea7c4d916688 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.424708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37832096-801b-4b2b-9eb6-dd396ac41c17 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.432523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bda0c-050d-46f4-808b-3045f6bcf107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.434108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bda0c-050d-46f4-808b-3045f6bcf107 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.434673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bda0c-050d-46f4-808b-3045f6bcf107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.435532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87bda0c-050d-46f4-808b-3045f6bcf107 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.436654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bda0c-050d-46f4-808b-3045f6bcf107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.436761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94a08a-9bff-4df3-9a24-ea7c4d916688 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.440257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87bda0c-050d-46f4-808b-3045f6bcf107 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.444994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f00357-52db-45a8-ab0f-d7cd2fe241f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.446370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f00357-52db-45a8-ab0f-d7cd2fe241f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.447143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f00357-52db-45a8-ab0f-d7cd2fe241f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.447781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f00357-52db-45a8-ab0f-d7cd2fe241f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.448935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f00357-52db-45a8-ab0f-d7cd2fe241f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.452631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f00357-52db-45a8-ab0f-d7cd2fe241f1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.469760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1079/2173] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:10.470968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f00357-52db-45a8-ab0f-d7cd2fe241f1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.483404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f00357-52db-45a8-ab0f-d7cd2fe241f1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.489356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bb59b-0cdb-49e9-b477-527e5eb6de1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.489672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1095/2174] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:10 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:10.490219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bb59b-0cdb-49e9-b477-527e5eb6de1b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.490658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bb59b-0cdb-49e9-b477-527e5eb6de1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.490983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bb59b-0cdb-49e9-b477-527e5eb6de1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.491751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bb59b-0cdb-49e9-b477-527e5eb6de1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.493760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bb59b-0cdb-49e9-b477-527e5eb6de1b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.495019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bda0c-050d-46f4-808b-3045f6bcf107 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.514871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1080/2175] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:10.516820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bb59b-0cdb-49e9-b477-527e5eb6de1b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.532523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bb59b-0cdb-49e9-b477-527e5eb6de1b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.579521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:10.606596 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87bda0c-050d-46f4-808b-3045f6bcf107 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.612999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b22c6-7d4c-48a3-a7a3-dc44816efc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.613700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b22c6-7d4c-48a3-a7a3-dc44816efc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.613982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b22c6-7d4c-48a3-a7a3-dc44816efc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.614258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b22c6-7d4c-48a3-a7a3-dc44816efc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.614930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b22c6-7d4c-48a3-a7a3-dc44816efc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.616602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b22c6-7d4c-48a3-a7a3-dc44816efc40 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.622972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8bfe3-1e7d-40d0-b71e-22e5842482f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.623740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8bfe3-1e7d-40d0-b71e-22e5842482f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.625726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8bfe3-1e7d-40d0-b71e-22e5842482f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.625726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8bfe3-1e7d-40d0-b71e-22e5842482f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.626091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8bfe3-1e7d-40d0-b71e-22e5842482f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.630214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8bfe3-1e7d-40d0-b71e-22e5842482f1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.633744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1096/2176] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:10.635215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b22c6-7d4c-48a3-a7a3-dc44816efc40 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.646003 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b22c6-7d4c-48a3-a7a3-dc44816efc40 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.652747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c30de-c79f-4910-8eda-e076f62311f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.654777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c30de-c79f-4910-8eda-e076f62311f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.655146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c30de-c79f-4910-8eda-e076f62311f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.655565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c30de-c79f-4910-8eda-e076f62311f1 admin admin] PATH_INFO: `/v3/projects/8b902097b23742a49a189b7b9aa822b9/users/d03ecc2999e74e72a29044b4b09940e3/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.669742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1081/2177] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:10 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:10.671068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8bfe3-1e7d-40d0-b71e-22e5842482f1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.684416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c30de-c79f-4910-8eda-e076f62311f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b902097b23742a49a189b7b9aa822b9, user_id=d03ecc2999e74e72a29044b4b09940e3, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.687270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c30de-c79f-4910-8eda-e076f62311f1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.689926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8bfe3-1e7d-40d0-b71e-22e5842482f1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.697779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ca638-f9ee-4713-b10a-f7d04cd5101d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.698951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca638-f9ee-4713-b10a-f7d04cd5101d admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.700585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca638-f9ee-4713-b10a-f7d04cd5101d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.701108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ca638-f9ee-4713-b10a-f7d04cd5101d admin admin] PATH_INFO: `/v3/projects/f75eb0c9b95443eea181e5ee642bee65/users/84d886f53b0d44a7910847bb8970a412/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.705812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1097/2178] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:10 2022] PUT /identity/v3/projects/8b902097b23742a49a189b7b9aa822b9/users/d03ecc2999e74e72a29044b4b09940e3/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:10.707184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c30de-c79f-4910-8eda-e076f62311f1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.737344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ca638-f9ee-4713-b10a-f7d04cd5101d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f75eb0c9b95443eea181e5ee642bee65, user_id=84d886f53b0d44a7910847bb8970a412, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.740195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ca638-f9ee-4713-b10a-f7d04cd5101d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.759270 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1082/2179] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:10 2022] PUT /identity/v3/projects/f75eb0c9b95443eea181e5ee642bee65/users/84d886f53b0d44a7910847bb8970a412/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:10.761326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba8ef5-f811-4e4e-b776-83f102eb4e70 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.763183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba8ef5-f811-4e4e-b776-83f102eb4e70 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.763526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba8ef5-f811-4e4e-b776-83f102eb4e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.803727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-bfba8ef5-f811-4e4e-b776-83f102eb4e70 None None] MFA Rules not processed for user `d03ecc2999e74e72a29044b4b09940e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:10.804784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c30de-c79f-4910-8eda-e076f62311f1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.805205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bfba8ef5-f811-4e4e-b776-83f102eb4e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:10.866739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:10.882040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686367a4-1050-47ad-8796-66f4aa1fc98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.883024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686367a4-1050-47ad-8796-66f4aa1fc98e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.883489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686367a4-1050-47ad-8796-66f4aa1fc98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.884227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686367a4-1050-47ad-8796-66f4aa1fc98e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.885255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686367a4-1050-47ad-8796-66f4aa1fc98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.889039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686367a4-1050-47ad-8796-66f4aa1fc98e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.903359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1098/2180] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:10.904736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa86dfe-8fec-4518-a556-79c82ad5a536 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.905057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa86dfe-8fec-4518-a556-79c82ad5a536 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.905333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa86dfe-8fec-4518-a556-79c82ad5a536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.912977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1083/2181] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:10.913913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba8ef5-f811-4e4e-b776-83f102eb4e70 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.944966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5aa86dfe-8fec-4518-a556-79c82ad5a536 None None] MFA Rules not processed for user `84d886f53b0d44a7910847bb8970a412`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:10.946655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5aa86dfe-8fec-4518-a556-79c82ad5a536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:10.948059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba8ef5-f811-4e4e-b776-83f102eb4e70 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:10.958193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:10.960278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:10.960658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:10.960959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:10.962126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:10.964422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:10.982063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1084/2182] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:10.983005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:10.984846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:11.087212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98220fc-262c-4f87-9ee4-82ae7b35b03f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:11.088113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1099/2183] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:11.089482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa86dfe-8fec-4518-a556-79c82ad5a536 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:11.101225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788954d-f133-4fda-a256-f10e4db8addd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:11.102075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788954d-f133-4fda-a256-f10e4db8addd service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.102490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788954d-f133-4fda-a256-f10e4db8addd service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.102684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788954d-f133-4fda-a256-f10e4db8addd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.111593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa86dfe-8fec-4518-a556-79c82ad5a536 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:11.114106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788954d-f133-4fda-a256-f10e4db8addd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:11.115761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788954d-f133-4fda-a256-f10e4db8addd service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:11.121541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f83aa-f723-4e9e-956e-fd3df3894703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:11.123509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f83aa-f723-4e9e-956e-fd3df3894703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.124664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f83aa-f723-4e9e-956e-fd3df3894703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.125649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f83aa-f723-4e9e-956e-fd3df3894703 admin admin] PATH_INFO: `/v3/projects/30be75cdd1f949f1894a5052cbd6755c/users/de09cae8e2204f7c91dc3764040588be/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.126302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1085/2184] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:11.127159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788954d-f133-4fda-a256-f10e4db8addd service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:11.142387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788954d-f133-4fda-a256-f10e4db8addd service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:11.149118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c2c04-f65f-4931-962b-be3571202dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:11.150131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c2c04-f65f-4931-962b-be3571202dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.150414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c2c04-f65f-4931-962b-be3571202dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.150689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c2c04-f65f-4931-962b-be3571202dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.151651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c2c04-f65f-4931-962b-be3571202dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:11.156306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c2c04-f65f-4931-962b-be3571202dc4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:11.160111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f83aa-f723-4e9e-956e-fd3df3894703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30be75cdd1f949f1894a5052cbd6755c, user_id=de09cae8e2204f7c91dc3764040588be, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:11.165902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f83aa-f723-4e9e-956e-fd3df3894703 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:11.191539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:11.197281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1100/2185] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:11 2022] PUT /identity/v3/projects/30be75cdd1f949f1894a5052cbd6755c/users/de09cae8e2204f7c91dc3764040588be/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:11.208631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1086/2186] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:11 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:11.213285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f83aa-f723-4e9e-956e-fd3df3894703 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:11.224901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b34ebc-ffd4-4dce-a8fb-e2944aebc532 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.225187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b34ebc-ffd4-4dce-a8fb-e2944aebc532 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.225483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b34ebc-ffd4-4dce-a8fb-e2944aebc532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.289941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-10b34ebc-ffd4-4dce-a8fb-e2944aebc532 None None] MFA Rules not processed for user `de09cae8e2204f7c91dc3764040588be`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:11.292338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-10b34ebc-ffd4-4dce-a8fb-e2944aebc532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:11.411345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f83aa-f723-4e9e-956e-fd3df3894703 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:11.440292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1087/2187] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:11.453537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b34ebc-ffd4-4dce-a8fb-e2944aebc532 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:11.459529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-10b34ebc-ffd4-4dce-a8fb-e2944aebc532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:11.557153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee8a38-f47a-407e-a105-37ed4fbd0afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:11.557949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee8a38-f47a-407e-a105-37ed4fbd0afa admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.558176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee8a38-f47a-407e-a105-37ed4fbd0afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.559245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee8a38-f47a-407e-a105-37ed4fbd0afa admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.560896 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee8a38-f47a-407e-a105-37ed4fbd0afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:11.563484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee8a38-f47a-407e-a105-37ed4fbd0afa admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:11.586544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:11.604513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b34ebc-ffd4-4dce-a8fb-e2944aebc532 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:11.609570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1101/2188] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:11 2022] POST /identity/v3/users => generated 294 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:11.610098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23fdec-ab79-4bc1-b797-7b4e2fb36798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:11.610919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23fdec-ab79-4bc1-b797-7b4e2fb36798 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.611521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23fdec-ab79-4bc1-b797-7b4e2fb36798 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.611911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23fdec-ab79-4bc1-b797-7b4e2fb36798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.615396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee8a38-f47a-407e-a105-37ed4fbd0afa admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:11.627263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23fdec-ab79-4bc1-b797-7b4e2fb36798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:11.627541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee8a38-f47a-407e-a105-37ed4fbd0afa admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:11.630175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23fdec-ab79-4bc1-b797-7b4e2fb36798 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:11.644835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d6740-aebc-4f2d-ba82-dc0e02adfb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:11.646734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6740-aebc-4f2d-ba82-dc0e02adfb04 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.648962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6740-aebc-4f2d-ba82-dc0e02adfb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.649869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6740-aebc-4f2d-ba82-dc0e02adfb04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.650085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1088/2189] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:11.652048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d6740-aebc-4f2d-ba82-dc0e02adfb04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:11.656949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d6740-aebc-4f2d-ba82-dc0e02adfb04 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:11.678378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1102/2190] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:11.683222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23fdec-ab79-4bc1-b797-7b4e2fb36798 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:11.706437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23fdec-ab79-4bc1-b797-7b4e2fb36798 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:11.723778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ea122-ce83-4d5b-9b13-2d6e9c3f56e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:11.726726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ea122-ce83-4d5b-9b13-2d6e9c3f56e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.727319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ea122-ce83-4d5b-9b13-2d6e9c3f56e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.728554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ea122-ce83-4d5b-9b13-2d6e9c3f56e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.731749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ea122-ce83-4d5b-9b13-2d6e9c3f56e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:11.737037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ea122-ce83-4d5b-9b13-2d6e9c3f56e6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:11.761989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1089/2191] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:11.767142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d6740-aebc-4f2d-ba82-dc0e02adfb04 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:11.795530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d6740-aebc-4f2d-ba82-dc0e02adfb04 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:11.803511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb72df-a3f3-4ff2-a510-8507933f8b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:11.804201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb72df-a3f3-4ff2-a510-8507933f8b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.804543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb72df-a3f3-4ff2-a510-8507933f8b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.804731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb72df-a3f3-4ff2-a510-8507933f8b9c admin admin] PATH_INFO: `/v3/projects/15664109288243539515b798bf33b977/users/c6c83e74fc3349e5bc07a19fef3f6684/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.848717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb72df-a3f3-4ff2-a510-8507933f8b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=15664109288243539515b798bf33b977, user_id=c6c83e74fc3349e5bc07a19fef3f6684, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:11.854863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb72df-a3f3-4ff2-a510-8507933f8b9c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:11.883053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1103/2192] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:11 2022] PUT /identity/v3/projects/15664109288243539515b798bf33b977/users/c6c83e74fc3349e5bc07a19fef3f6684/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:11.908353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2a726-43e6-40e7-9c53-0570485bc8a0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:11.908829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2a726-43e6-40e7-9c53-0570485bc8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:11.909277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2a726-43e6-40e7-9c53-0570485bc8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:11.963471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-07d2a726-43e6-40e7-9c53-0570485bc8a0 None None] MFA Rules not processed for user `c6c83e74fc3349e5bc07a19fef3f6684`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:11.967612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-07d2a726-43e6-40e7-9c53-0570485bc8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:12.157616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1090/2193] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:12.165132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb72df-a3f3-4ff2-a510-8507933f8b9c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:12.291310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb72df-a3f3-4ff2-a510-8507933f8b9c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:12.299458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd74f7e-3680-45c5-96f3-dd7e181cdea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:12.300193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74f7e-3680-45c5-96f3-dd7e181cdea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:12.300483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74f7e-3680-45c5-96f3-dd7e181cdea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:12.300761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd74f7e-3680-45c5-96f3-dd7e181cdea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:12.301424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd74f7e-3680-45c5-96f3-dd7e181cdea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:12.304127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd74f7e-3680-45c5-96f3-dd7e181cdea5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:12.343717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1104/2194] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:12 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:12.352108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2a726-43e6-40e7-9c53-0570485bc8a0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:12.405444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2a726-43e6-40e7-9c53-0570485bc8a0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:12.413685 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:12.517603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f670db4-a892-4fbb-bb54-8456f016fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:12.518651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f670db4-a892-4fbb-bb54-8456f016fff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:12.518844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f670db4-a892-4fbb-bb54-8456f016fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:12.519191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f670db4-a892-4fbb-bb54-8456f016fff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:12.519954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f670db4-a892-4fbb-bb54-8456f016fff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:12.522631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f670db4-a892-4fbb-bb54-8456f016fff6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:12.555464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1091/2195] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:12 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:12.562066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd74f7e-3680-45c5-96f3-dd7e181cdea5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:12.578147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd74f7e-3680-45c5-96f3-dd7e181cdea5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:12.591959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b3c10-04ec-4a44-8ef7-42248255aa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:12.593412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b3c10-04ec-4a44-8ef7-42248255aa84 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:12.593412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b3c10-04ec-4a44-8ef7-42248255aa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:12.594253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b3c10-04ec-4a44-8ef7-42248255aa84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:12.595811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8b3c10-04ec-4a44-8ef7-42248255aa84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:12.597557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8b3c10-04ec-4a44-8ef7-42248255aa84 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:12.614659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1105/2196] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:12.622215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f670db4-a892-4fbb-bb54-8456f016fff6 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:12.642249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f670db4-a892-4fbb-bb54-8456f016fff6 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:12.654375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdc138-bf85-4ab1-9c83-7902632489f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:12.656154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdc138-bf85-4ab1-9c83-7902632489f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:12.656510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdc138-bf85-4ab1-9c83-7902632489f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:12.657003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdc138-bf85-4ab1-9c83-7902632489f4 admin admin] PATH_INFO: `/v3/projects/d7d67da618e048408da1c1a290da92bf/users/b0f234116ffb4119b676feac40049a8f/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:12.695457 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdc138-bf85-4ab1-9c83-7902632489f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d67da618e048408da1c1a290da92bf, user_id=b0f234116ffb4119b676feac40049a8f, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:12.704339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdc138-bf85-4ab1-9c83-7902632489f4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:12.751667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1092/2197] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:12 2022] PUT /identity/v3/projects/d7d67da618e048408da1c1a290da92bf/users/b0f234116ffb4119b676feac40049a8f/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:12.753406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b3c10-04ec-4a44-8ef7-42248255aa84 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:12.781577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b3c10-04ec-4a44-8ef7-42248255aa84 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:12.893012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8fa66-6fe1-406b-ab09-b6e81629e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:12.898887 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8fa66-6fe1-406b-ab09-b6e81629e670 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:12.900163 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8fa66-6fe1-406b-ab09-b6e81629e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:12.906802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8fa66-6fe1-406b-ab09-b6e81629e670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:12.907980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8fa66-6fe1-406b-ab09-b6e81629e670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:12.910886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8fa66-6fe1-406b-ab09-b6e81629e670 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:12.997274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdc138-bf85-4ab1-9c83-7902632489f4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:13.000264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1106/2198] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:13.030683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdc138-bf85-4ab1-9c83-7902632489f4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:13.052665 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132655a6-8dbf-4eea-9a81-2fc82fff674a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:13.053376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132655a6-8dbf-4eea-9a81-2fc82fff674a admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:13.053769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132655a6-8dbf-4eea-9a81-2fc82fff674a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:13.054035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132655a6-8dbf-4eea-9a81-2fc82fff674a admin admin] PATH_INFO: `/v3/projects/d7d67da618e048408da1c1a290da92bf/users/b0f234116ffb4119b676feac40049a8f/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:13.056592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132655a6-8dbf-4eea-9a81-2fc82fff674a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d67da618e048408da1c1a290da92bf, user_id=b0f234116ffb4119b676feac40049a8f, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:13.060966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132655a6-8dbf-4eea-9a81-2fc82fff674a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:13.076808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1093/2199] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:12 2022] PUT /identity/v3/projects/d7d67da618e048408da1c1a290da92bf/users/b0f234116ffb4119b676feac40049a8f/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:13.096112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449548-f9ab-464e-acbf-037eebf37a9d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:13.096935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449548-f9ab-464e-acbf-037eebf37a9d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:13.098340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac449548-f9ab-464e-acbf-037eebf37a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:13.197866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ac449548-f9ab-464e-acbf-037eebf37a9d None None] MFA Rules not processed for user `b0f234116ffb4119b676feac40049a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:13.201909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ac449548-f9ab-464e-acbf-037eebf37a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:13.234717 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:13.392461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1107/2200] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:13.459661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132655a6-8dbf-4eea-9a81-2fc82fff674a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:13.571201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132655a6-8dbf-4eea-9a81-2fc82fff674a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:13.583837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8948564-4cad-4f16-868a-35b21788d189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:13.584907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8948564-4cad-4f16-868a-35b21788d189 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:13.585442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8948564-4cad-4f16-868a-35b21788d189 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:13.585958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8948564-4cad-4f16-868a-35b21788d189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:13.600960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8948564-4cad-4f16-868a-35b21788d189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:13.603054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8948564-4cad-4f16-868a-35b21788d189 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:13.615984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1094/2201] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:14.634891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac449548-f9ab-464e-acbf-037eebf37a9d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:14.646582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac449548-f9ab-464e-acbf-037eebf37a9d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:14.651168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:14.652716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:14.653172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:14.653522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:14.750003 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:14.752042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:14.768341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1108/2202] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:18.816356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8948564-4cad-4f16-868a-35b21788d189 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:18.831740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:18.834945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:18.972559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8948564-4cad-4f16-868a-35b21788d189 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:18.982099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:18.983259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:18.983675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:18.984143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] PATH_INFO: `/v3/users/d03ecc2999e74e72a29044b4b09940e3` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:18.986272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03ecc2999e74e72a29044b4b09940e3)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:18.990217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:18.995616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:19.007787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a7ea-8aed-4aea-9d56-784bfbab5a8b service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:19.017156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:19.018899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:19.018899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:19.019605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] PATH_INFO: `/v3/users/7c3cbb6c23e2487e8d798e87f8310989` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:19.021499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c3cbb6c23e2487e8d798e87f8310989)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:19.025133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:19.071596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03ecc2999e74e72a29044b4b09940e3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.072512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03ecc2999e74e72a29044b4b09940e3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.073284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03ecc2999e74e72a29044b4b09940e3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.086035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03ecc2999e74e72a29044b4b09940e3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.106594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fa1e8541-e82a-4bdc-8d35-1bf749b85d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03ecc2999e74e72a29044b4b09940e3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.128052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1095/2203] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:18 2022] DELETE /identity/v3/users/d03ecc2999e74e72a29044b4b09940e3 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:19.137553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27ec1-737d-4c45-9ef1-2fb080bdd61d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:19.137853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27ec1-737d-4c45-9ef1-2fb080bdd61d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:19.138265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def27ec1-737d-4c45-9ef1-2fb080bdd61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:19.140119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c3cbb6c23e2487e8d798e87f8310989'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.145250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:19.164923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-def27ec1-737d-4c45-9ef1-2fb080bdd61d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:19.167151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-def27ec1-737d-4c45-9ef1-2fb080bdd61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:19.169406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c3cbb6c23e2487e8d798e87f8310989'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.170708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c3cbb6c23e2487e8d798e87f8310989'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.185635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c3cbb6c23e2487e8d798e87f8310989'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.200477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8fae3e4f-0763-45c8-84ce-bb9585b66284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c3cbb6c23e2487e8d798e87f8310989'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:19.218209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1109/2204] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:18 2022] DELETE /identity/v3/users/7c3cbb6c23e2487e8d798e87f8310989 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:19.224988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66458f-1f30-48e9-93ba-fd512d7f717a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:19.225433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66458f-1f30-48e9-93ba-fd512d7f717a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:19.225842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66458f-1f30-48e9-93ba-fd512d7f717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:19.259786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aa66458f-1f30-48e9-93ba-fd512d7f717a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:19.261038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aa66458f-1f30-48e9-93ba-fd512d7f717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:19.262974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1096/2205] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:19.284257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def27ec1-737d-4c45-9ef1-2fb080bdd61d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:19.287342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-def27ec1-737d-4c45-9ef1-2fb080bdd61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:19.382797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1110/2206] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:19.385767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def27ec1-737d-4c45-9ef1-2fb080bdd61d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:19.393895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66458f-1f30-48e9-93ba-fd512d7f717a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:19.395627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aa66458f-1f30-48e9-93ba-fd512d7f717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:19.432206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:19.458487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:19.459147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:19.459471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:19.460090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:19.472233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:19.473751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:19.474852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66458f-1f30-48e9-93ba-fd512d7f717a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:19.478839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:19.479468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:19.479752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:19.481050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:19.485411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1097/2207] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:19.492526 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:19.494088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:19.506056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1111/2208] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:19.861915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:19.865991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:19.868422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:19.870818 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:19.890395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baff25-a8e8-4ad2-ac2f-23e6f4b04a32 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:19.896244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:19.896920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:19.897187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:19.897631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] PATH_INFO: `/v3/projects/8b902097b23742a49a189b7b9aa822b9` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:19.899422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b902097b23742a49a189b7b9aa822b9)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:19.903548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:19.933035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde00de-e505-41f9-82ff-33c91fc66db8 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:19.946556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:19.948057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:19.949370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:19.950305 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] PATH_INFO: `/v3/projects/b4e42e3a5abe462c80c6fd12973618a3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:19.952928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e42e3a5abe462c80c6fd12973618a3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:19.957272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:20.023976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b902097b23742a49a189b7b9aa822b9', 'request_id': 'req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.036007 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b902097b23742a49a189b7b9aa822b9', 'request_id': 'req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.044921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b902097b23742a49a189b7b9aa822b9', 'request_id': 'req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.053803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e42e3a5abe462c80c6fd12973618a3', 'request_id': 'req-d3871918-1b9c-40ea-9912-c6dcd011d172'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.062875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] The token cache is being invalidate because project 8b902097b23742a49a189b7b9aa822b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:20.063360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.064211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-0580b3a3-5f1b-43ce-8cd8-515eb6ec3d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.064562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e42e3a5abe462c80c6fd12973618a3', 'request_id': 'req-d3871918-1b9c-40ea-9912-c6dcd011d172'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.066119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1098/2209] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:19 2022] DELETE /identity/v3/projects/8b902097b23742a49a189b7b9aa822b9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:20.071645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3087d-e454-4943-bca7-c603b9fbb79e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.072104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3087d-e454-4943-bca7-c603b9fbb79e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.072560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3087d-e454-4943-bca7-c603b9fbb79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.079708 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e42e3a5abe462c80c6fd12973618a3', 'request_id': 'req-d3871918-1b9c-40ea-9912-c6dcd011d172'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.081941 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:20.099982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-90b3087d-e454-4943-bca7-c603b9fbb79e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:20.101240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-90b3087d-e454-4943-bca7-c603b9fbb79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:20.106174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] The token cache is being invalidate because project b4e42e3a5abe462c80c6fd12973618a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:20.107061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.108702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.111598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1112/2210] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:19 2022] DELETE /identity/v3/projects/b4e42e3a5abe462c80c6fd12973618a3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:20.117829 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:20.120275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:20.217514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1099/2211] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:20.225315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040a144-4cee-4cb3-9cb1-474460ee32f1 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.225824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040a144-4cee-4cb3-9cb1-474460ee32f1 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.226300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040a144-4cee-4cb3-9cb1-474460ee32f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.233982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3871918-1b9c-40ea-9912-c6dcd011d172 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:20.238875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:20.239724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.240019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.240307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] PATH_INFO: `/v3/users/a28a0c41c43246afa22643b472e70cf7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.241546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a28a0c41c43246afa22643b472e70cf7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:20.243557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:20.261094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8040a144-4cee-4cb3-9cb1-474460ee32f1 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:20.263433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8040a144-4cee-4cb3-9cb1-474460ee32f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:20.279114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1100/2212] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:20.284218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040a144-4cee-4cb3-9cb1-474460ee32f1 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:20.296152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040a144-4cee-4cb3-9cb1-474460ee32f1 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:20.301482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:20.302117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.302429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.302707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.303378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:20.303712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:20.305378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:20.317734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a28a0c41c43246afa22643b472e70cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.320972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1101/2213] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:32:20.329252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:20.338919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a28a0c41c43246afa22643b472e70cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.340334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a28a0c41c43246afa22643b472e70cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.354351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a28a0c41c43246afa22643b472e70cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.355875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1e6fc-4ede-4afd-8d4c-1ea99e7a7fcb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:20.363650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a51a39-00dd-44af-aad1-8058ae7f0237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:20.364661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a51a39-00dd-44af-aad1-8058ae7f0237 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.365133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a51a39-00dd-44af-aad1-8058ae7f0237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.365628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a51a39-00dd-44af-aad1-8058ae7f0237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.366755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a51a39-00dd-44af-aad1-8058ae7f0237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:20.367390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a28a0c41c43246afa22643b472e70cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:20.369318 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a51a39-00dd-44af-aad1-8058ae7f0237 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:20.385122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1113/2214] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:20 2022] DELETE /identity/v3/users/a28a0c41c43246afa22643b472e70cf7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:20.437813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1102/2215] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:20 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:20.444380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:20.447459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:20.557855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f90c6-8c88-443c-a8d9-1fa1a2bbb918 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:20.579110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:20.629604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a70ef7-7e9f-4c6b-ad8b-f1e98dd1c7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:20.630484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70ef7-7e9f-4c6b-ad8b-f1e98dd1c7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.630897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70ef7-7e9f-4c6b-ad8b-f1e98dd1c7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.631298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a70ef7-7e9f-4c6b-ad8b-f1e98dd1c7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.632144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a70ef7-7e9f-4c6b-ad8b-f1e98dd1c7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:20.634251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a70ef7-7e9f-4c6b-ad8b-f1e98dd1c7a4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:20.662557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1114/2216] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:20 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:20.667037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a51a39-00dd-44af-aad1-8058ae7f0237 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:20.679338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a51a39-00dd-44af-aad1-8058ae7f0237 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:20.684705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6218f50-9002-463d-b88d-03502283af86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:20.685667 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6218f50-9002-463d-b88d-03502283af86 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.686162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6218f50-9002-463d-b88d-03502283af86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.686569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6218f50-9002-463d-b88d-03502283af86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.687355 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6218f50-9002-463d-b88d-03502283af86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:20.689141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6218f50-9002-463d-b88d-03502283af86 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:20.701766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1103/2217] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:20.705982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a70ef7-7e9f-4c6b-ad8b-f1e98dd1c7a4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:20.719351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a70ef7-7e9f-4c6b-ad8b-f1e98dd1c7a4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:20.724260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5560e-d829-4ad9-b913-f1a99dbb94dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:20.724903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5560e-d829-4ad9-b913-f1a99dbb94dc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.725186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5560e-d829-4ad9-b913-f1a99dbb94dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.725490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5560e-d829-4ad9-b913-f1a99dbb94dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.726386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5560e-d829-4ad9-b913-f1a99dbb94dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:20.728205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5560e-d829-4ad9-b913-f1a99dbb94dc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:20.740350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1115/2218] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:20.760782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6218f50-9002-463d-b88d-03502283af86 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:20.782202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6218f50-9002-463d-b88d-03502283af86 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:20.787178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:20.787868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.788148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.788416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] PATH_INFO: `/v3/projects/6e3ee64f20864e4daffcccbd8a7b2be5/users/7fcee04e4b2e478fa47b77de20f6658b/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.798389 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:20.807190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3ee64f20864e4daffcccbd8a7b2be5, user_id=7fcee04e4b2e478fa47b77de20f6658b, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:20.812146 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:20.836532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1104/2219] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:20 2022] PUT /identity/v3/projects/6e3ee64f20864e4daffcccbd8a7b2be5/users/7fcee04e4b2e478fa47b77de20f6658b/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:20.848348 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04335075-ec68-4c22-90ef-0fbf51268b65 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:20.848637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04335075-ec68-4c22-90ef-0fbf51268b65 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:20.848927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04335075-ec68-4c22-90ef-0fbf51268b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:20.889377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-04335075-ec68-4c22-90ef-0fbf51268b65 None None] MFA Rules not processed for user `7fcee04e4b2e478fa47b77de20f6658b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:20.891575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-04335075-ec68-4c22-90ef-0fbf51268b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:20.926774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:20.929829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:20.994763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1116/2220] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:21.003125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04335075-ec68-4c22-90ef-0fbf51268b65 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.027072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7399e6-02d1-432f-b629-ba0d802624c8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.038089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.039019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.039407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.039858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] PATH_INFO: `/v3/projects/3836cab5341d4a6fbd53970562a58819` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.041091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3836cab5341d4a6fbd53970562a58819)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.044231 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.079037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04335075-ec68-4c22-90ef-0fbf51268b65 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.084260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfd2b-dacf-4808-8ab8-8c8789c670f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.084891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfd2b-dacf-4808-8ab8-8c8789c670f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.085189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfd2b-dacf-4808-8ab8-8c8789c670f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.085490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bfd2b-dacf-4808-8ab8-8c8789c670f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.086153 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bfd2b-dacf-4808-8ab8-8c8789c670f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.089118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bfd2b-dacf-4808-8ab8-8c8789c670f5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.119432 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3836cab5341d4a6fbd53970562a58819', 'request_id': 'req-95c3314c-f289-4da8-8f01-38c522ab5e03'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:21.128683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1117/2221] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:21.129074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3836cab5341d4a6fbd53970562a58819', 'request_id': 'req-95c3314c-f289-4da8-8f01-38c522ab5e03'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:21.132922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfd2b-dacf-4808-8ab8-8c8789c670f5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.140490 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3836cab5341d4a6fbd53970562a58819', 'request_id': 'req-95c3314c-f289-4da8-8f01-38c522ab5e03'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:21.154340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:21.157566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] The token cache is being invalidate because project 3836cab5341d4a6fbd53970562a58819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:21.157869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:21.158582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-95c3314c-f289-4da8-8f01-38c522ab5e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:21.160328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1105/2222] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:20 2022] DELETE /identity/v3/projects/3836cab5341d4a6fbd53970562a58819 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:21.166063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5319-d809-4b8c-bc9d-7099d2eff99b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.166386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5319-d809-4b8c-bc9d-7099d2eff99b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.166855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a5319-d809-4b8c-bc9d-7099d2eff99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.192855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d03a5319-d809-4b8c-bc9d-7099d2eff99b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:21.194137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d03a5319-d809-4b8c-bc9d-7099d2eff99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:21.230502 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bfd2b-dacf-4808-8ab8-8c8789c670f5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.235752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.236570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.236857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.237652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.238350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.240536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.271194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1118/2223] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:21.272634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.275607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:21.287722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1106/2224] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:21.296210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fa66b-c9c9-4d79-934f-69543573e974 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.296503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fa66b-c9c9-4d79-934f-69543573e974 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.296784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fa66b-c9c9-4d79-934f-69543573e974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.297762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05503-1747-4f42-a1b6-a153efea6e60 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.303171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0a16d-6240-48db-ba1e-6c634c9e249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.303823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a16d-6240-48db-ba1e-6c634c9e249d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.304121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a16d-6240-48db-ba1e-6c634c9e249d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.304393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0a16d-6240-48db-ba1e-6c634c9e249d admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.305139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0a16d-6240-48db-ba1e-6c634c9e249d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.307316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b0a16d-6240-48db-ba1e-6c634c9e249d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.319884 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-723fa66b-c9c9-4d79-934f-69543573e974 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:21.324629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1119/2225] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:21.324811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-723fa66b-c9c9-4d79-934f-69543573e974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:21.326197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0a16d-6240-48db-ba1e-6c634c9e249d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.330589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1107/2226] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:21.334900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fa66b-c9c9-4d79-934f-69543573e974 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.338204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0a16d-6240-48db-ba1e-6c634c9e249d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.346514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fa66b-c9c9-4d79-934f-69543573e974 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.349188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a940cba-c636-4ff0-96a8-e38931b12125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.349851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a940cba-c636-4ff0-96a8-e38931b12125 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.350140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a940cba-c636-4ff0-96a8-e38931b12125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.350452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a940cba-c636-4ff0-96a8-e38931b12125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.351120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a940cba-c636-4ff0-96a8-e38931b12125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.352759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a940cba-c636-4ff0-96a8-e38931b12125 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.353812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.355871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.355871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.355871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.355871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:21.355871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.357186 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.366003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:21.378319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1120/2227] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:21.381699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1108/2228] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:21.383833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a940cba-c636-4ff0-96a8-e38931b12125 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.385282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.395998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a940cba-c636-4ff0-96a8-e38931b12125 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.400878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9138e0-7812-4c99-b2ff-3a765c3eadc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.401013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22277939-adfa-4964-b7ac-04128081f54a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.401557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9138e0-7812-4c99-b2ff-3a765c3eadc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.401948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9138e0-7812-4c99-b2ff-3a765c3eadc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.402955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9138e0-7812-4c99-b2ff-3a765c3eadc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.403767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9138e0-7812-4c99-b2ff-3a765c3eadc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.405808 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9138e0-7812-4c99-b2ff-3a765c3eadc7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.408846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131eb37-46a9-4bf2-b3d3-e0d1a0826c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.410037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131eb37-46a9-4bf2-b3d3-e0d1a0826c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.410539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131eb37-46a9-4bf2-b3d3-e0d1a0826c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.411415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131eb37-46a9-4bf2-b3d3-e0d1a0826c7c admin admin] PATH_INFO: `/v3/projects/f94197d784eb4a66999063169e0a4562/users/edcd85bc334f41719b0713c80642a55a/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.431935 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131eb37-46a9-4bf2-b3d3-e0d1a0826c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94197d784eb4a66999063169e0a4562, user_id=edcd85bc334f41719b0713c80642a55a, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.435483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131eb37-46a9-4bf2-b3d3-e0d1a0826c7c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.443072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1121/2229] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:21.449312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9138e0-7812-4c99-b2ff-3a765c3eadc7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.455753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1109/2230] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:21 2022] PUT /identity/v3/projects/f94197d784eb4a66999063169e0a4562/users/edcd85bc334f41719b0713c80642a55a/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:21.468263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0396d-bbc9-41b8-9ec9-da45087f0a72 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.468663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0396d-bbc9-41b8-9ec9-da45087f0a72 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.469094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0396d-bbc9-41b8-9ec9-da45087f0a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.508810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a1a0396d-bbc9-41b8-9ec9-da45087f0a72 None None] MFA Rules not processed for user `edcd85bc334f41719b0713c80642a55a`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:21.510440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a1a0396d-bbc9-41b8-9ec9-da45087f0a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:21.541155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9138e0-7812-4c99-b2ff-3a765c3eadc7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.612536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1110/2231] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:21.617883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0396d-bbc9-41b8-9ec9-da45087f0a72 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.631415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ebd36-537e-4b95-884f-ea56a2dc849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.632098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ebd36-537e-4b95-884f-ea56a2dc849a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.632563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ebd36-537e-4b95-884f-ea56a2dc849a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.632854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ebd36-537e-4b95-884f-ea56a2dc849a admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.633609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ebd36-537e-4b95-884f-ea56a2dc849a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.636262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ebd36-537e-4b95-884f-ea56a2dc849a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.650697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:21.675026 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1122/2232] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:21.682593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ebd36-537e-4b95-884f-ea56a2dc849a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.698373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0396d-bbc9-41b8-9ec9-da45087f0a72 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.701684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ebd36-537e-4b95-884f-ea56a2dc849a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.705204 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919a7ac-3a9a-470b-b977-6b02f91455da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.706046 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919a7ac-3a9a-470b-b977-6b02f91455da admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.706442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919a7ac-3a9a-470b-b977-6b02f91455da admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.706801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919a7ac-3a9a-470b-b977-6b02f91455da admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.707649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919a7ac-3a9a-470b-b977-6b02f91455da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.709966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b6688-4c7a-4b57-8869-870971ff4f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.710196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2919a7ac-3a9a-470b-b977-6b02f91455da admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.710946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b6688-4c7a-4b57-8869-870971ff4f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.711416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b6688-4c7a-4b57-8869-870971ff4f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.711919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b6688-4c7a-4b57-8869-870971ff4f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.713098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b6688-4c7a-4b57-8869-870971ff4f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.716281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b6688-4c7a-4b57-8869-870971ff4f49 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.734245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1123/2233] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:21.739695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b6688-4c7a-4b57-8869-870971ff4f49 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.749828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1111/2234] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:21.750975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919a7ac-3a9a-470b-b977-6b02f91455da admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.752843 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b6688-4c7a-4b57-8869-870971ff4f49 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.813349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:21.832966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919a7ac-3a9a-470b-b977-6b02f91455da admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.839501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3662e5-4c1e-4511-b3fd-9a1709e5293a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.840144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3662e5-4c1e-4511-b3fd-9a1709e5293a admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.840590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3662e5-4c1e-4511-b3fd-9a1709e5293a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.840984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3662e5-4c1e-4511-b3fd-9a1709e5293a admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.841976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3662e5-4c1e-4511-b3fd-9a1709e5293a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.842512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f1676-e6b1-4514-958e-50abdbce764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.843143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f1676-e6b1-4514-958e-50abdbce764f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.843597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f1676-e6b1-4514-958e-50abdbce764f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.843879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f1676-e6b1-4514-958e-50abdbce764f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.844069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3662e5-4c1e-4511-b3fd-9a1709e5293a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.844544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f1676-e6b1-4514-958e-50abdbce764f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.846331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f1676-e6b1-4514-958e-50abdbce764f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.859580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1124/2235] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:21.864643 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f1676-e6b1-4514-958e-50abdbce764f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.870454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1112/2236] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:21.876094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3662e5-4c1e-4511-b3fd-9a1709e5293a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.883283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f1676-e6b1-4514-958e-50abdbce764f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.889819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fea92-c33f-4475-b7cd-1c7c474d3601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.890654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fea92-c33f-4475-b7cd-1c7c474d3601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.890938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fea92-c33f-4475-b7cd-1c7c474d3601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.891217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fea92-c33f-4475-b7cd-1c7c474d3601 admin admin] PATH_INFO: `/v3/projects/9c9e6933eeab476497bd725daff9be90/users/9b04ae2d0e8047d7a236422a1502f86c/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.891315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3662e5-4c1e-4511-b3fd-9a1709e5293a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.897175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3caaf8f-5224-48a0-be38-b8edd8f40c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:21.897872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caaf8f-5224-48a0-be38-b8edd8f40c0b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.898163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caaf8f-5224-48a0-be38-b8edd8f40c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.898453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3caaf8f-5224-48a0-be38-b8edd8f40c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.899206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3caaf8f-5224-48a0-be38-b8edd8f40c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.900900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3caaf8f-5224-48a0-be38-b8edd8f40c0b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.911175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fea92-c33f-4475-b7cd-1c7c474d3601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9e6933eeab476497bd725daff9be90, user_id=9b04ae2d0e8047d7a236422a1502f86c, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:21.912951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1113/2237] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:21.914878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fea92-c33f-4475-b7cd-1c7c474d3601 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:21.917181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3caaf8f-5224-48a0-be38-b8edd8f40c0b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:21.932425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3caaf8f-5224-48a0-be38-b8edd8f40c0b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:21.935671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1125/2238] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:21 2022] PUT /identity/v3/projects/9c9e6933eeab476497bd725daff9be90/users/9b04ae2d0e8047d7a236422a1502f86c/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:21.947856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a3b83-fc0f-4376-935e-17ff31f0473a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:21.948370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a3b83-fc0f-4376-935e-17ff31f0473a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:21.949549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a3b83-fc0f-4376-935e-17ff31f0473a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:21.989854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-984a3b83-fc0f-4376-935e-17ff31f0473a None None] MFA Rules not processed for user `9b04ae2d0e8047d7a236422a1502f86c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:21.991150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-984a3b83-fc0f-4376-935e-17ff31f0473a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:22.021358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d6aa-f56c-4072-b020-1dea7477bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.021864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d6aa-f56c-4072-b020-1dea7477bcf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.022152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d6aa-f56c-4072-b020-1dea7477bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.025701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d6aa-f56c-4072-b020-1dea7477bcf3 admin admin] PATH_INFO: `/v3/projects/e905cdccc5c241c09832623a7396868e/users/c7a18ca4a3944d21812252a10b66ba42/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.044671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64d6aa-f56c-4072-b020-1dea7477bcf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e905cdccc5c241c09832623a7396868e, user_id=c7a18ca4a3944d21812252a10b66ba42, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.047954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64d6aa-f56c-4072-b020-1dea7477bcf3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.068622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1114/2239] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:21 2022] PUT /identity/v3/projects/e905cdccc5c241c09832623a7396868e/users/c7a18ca4a3944d21812252a10b66ba42/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:22.075144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d6aa-f56c-4072-b020-1dea7477bcf3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.087357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1126/2240] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:22.093909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a3b83-fc0f-4376-935e-17ff31f0473a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.108605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a3b83-fc0f-4376-935e-17ff31f0473a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.194416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d6aa-f56c-4072-b020-1dea7477bcf3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.202756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d09c43-0250-4b5e-8b6f-f25fd3d0eb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.203680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d09c43-0250-4b5e-8b6f-f25fd3d0eb71 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.203920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d09c43-0250-4b5e-8b6f-f25fd3d0eb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.204375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d09c43-0250-4b5e-8b6f-f25fd3d0eb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.205505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d09c43-0250-4b5e-8b6f-f25fd3d0eb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.207995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d09c43-0250-4b5e-8b6f-f25fd3d0eb71 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.208430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:22.232209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010f9ea-e7a0-462c-93e0-d7d44093825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.238200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f9ea-e7a0-462c-93e0-d7d44093825f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.238479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1115/2241] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:22.238623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f9ea-e7a0-462c-93e0-d7d44093825f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.238623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010f9ea-e7a0-462c-93e0-d7d44093825f admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.238623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010f9ea-e7a0-462c-93e0-d7d44093825f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.238623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010f9ea-e7a0-462c-93e0-d7d44093825f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.240209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d09c43-0250-4b5e-8b6f-f25fd3d0eb71 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.260055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d09c43-0250-4b5e-8b6f-f25fd3d0eb71 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.268202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff70e8-1939-4567-938d-2f4f994be0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.268963 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff70e8-1939-4567-938d-2f4f994be0af admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.269378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff70e8-1939-4567-938d-2f4f994be0af admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.269700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff70e8-1939-4567-938d-2f4f994be0af admin admin] PATH_INFO: `/v3/projects/e905cdccc5c241c09832623a7396868e/users/c7a18ca4a3944d21812252a10b66ba42/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.272134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff70e8-1939-4567-938d-2f4f994be0af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e905cdccc5c241c09832623a7396868e, user_id=c7a18ca4a3944d21812252a10b66ba42, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.274543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1127/2242] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:22 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:22.274882 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff70e8-1939-4567-938d-2f4f994be0af admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.279723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010f9ea-e7a0-462c-93e0-d7d44093825f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.288298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1116/2243] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:22 2022] PUT /identity/v3/projects/e905cdccc5c241c09832623a7396868e/users/c7a18ca4a3944d21812252a10b66ba42/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:22.311673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade399-f460-4506-9d06-39db4bb33263 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.312060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade399-f460-4506-9d06-39db4bb33263 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.312354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eade399-f460-4506-9d06-39db4bb33263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.377393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010f9ea-e7a0-462c-93e0-d7d44093825f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.377917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3eade399-f460-4506-9d06-39db4bb33263 None None] MFA Rules not processed for user `c7a18ca4a3944d21812252a10b66ba42`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:22.379762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3eade399-f460-4506-9d06-39db4bb33263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:22.406257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:22.456621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22520de7-7733-416a-bf28-94c0cf4dbad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.457257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22520de7-7733-416a-bf28-94c0cf4dbad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.457583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22520de7-7733-416a-bf28-94c0cf4dbad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.457854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22520de7-7733-416a-bf28-94c0cf4dbad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.458500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22520de7-7733-416a-bf28-94c0cf4dbad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.460492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22520de7-7733-416a-bf28-94c0cf4dbad3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.488786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1128/2244] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:22 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:22.490008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22520de7-7733-416a-bf28-94c0cf4dbad3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.495271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1117/2245] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:22.503677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22520de7-7733-416a-bf28-94c0cf4dbad3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.510008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcb91d-58dd-4027-9190-118b317c818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.510697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcb91d-58dd-4027-9190-118b317c818c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.510987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcb91d-58dd-4027-9190-118b317c818c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.511963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcb91d-58dd-4027-9190-118b317c818c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.513066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcb91d-58dd-4027-9190-118b317c818c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.514985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedcb91d-58dd-4027-9190-118b317c818c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.515168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eade399-f460-4506-9d06-39db4bb33263 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.516740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3eade399-f460-4506-9d06-39db4bb33263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:22.530586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1129/2246] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:22.532078 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcb91d-58dd-4027-9190-118b317c818c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.543739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcb91d-58dd-4027-9190-118b317c818c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.548761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2c94-5e77-464c-8eca-8eb9d7cb6b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.550135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2c94-5e77-464c-8eca-8eb9d7cb6b72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.550135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2c94-5e77-464c-8eca-8eb9d7cb6b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.550135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2c94-5e77-464c-8eca-8eb9d7cb6b72 admin admin] PATH_INFO: `/v3/projects/6315ecc51b8a4bc5aaa2b8ec65003a11/users/1d66ea357e9c42dd98d3fabf328718c8/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.567765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba2c94-5e77-464c-8eca-8eb9d7cb6b72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6315ecc51b8a4bc5aaa2b8ec65003a11, user_id=1d66ea357e9c42dd98d3fabf328718c8, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.570622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba2c94-5e77-464c-8eca-8eb9d7cb6b72 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.586522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1130/2247] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:22 2022] PUT /identity/v3/projects/6315ecc51b8a4bc5aaa2b8ec65003a11/users/1d66ea357e9c42dd98d3fabf328718c8/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:22.591359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2c94-5e77-464c-8eca-8eb9d7cb6b72 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.603383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eade399-f460-4506-9d06-39db4bb33263 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.635200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:22.680601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.681752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.682193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.682645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.683000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2c94-5e77-464c-8eca-8eb9d7cb6b72 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.686716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:22.691765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d3680-4d63-4856-8d98-564a481a9fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.692735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d3680-4d63-4856-8d98-564a481a9fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.693192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d3680-4d63-4856-8d98-564a481a9fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.693699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d3680-4d63-4856-8d98-564a481a9fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.694815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d3680-4d63-4856-8d98-564a481a9fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.697833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d3680-4d63-4856-8d98-564a481a9fc2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.720702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1131/2248] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:22.722257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d3680-4d63-4856-8d98-564a481a9fc2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.741845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d3680-4d63-4856-8d98-564a481a9fc2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:22.748415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba8dd6-fb46-49db-9dc9-bb29e61f71d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:22.749180 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8dd6-fb46-49db-9dc9-bb29e61f71d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.749504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8dd6-fb46-49db-9dc9-bb29e61f71d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.750047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba8dd6-fb46-49db-9dc9-bb29e61f71d0 admin admin] PATH_INFO: `/v3/projects/6315ecc51b8a4bc5aaa2b8ec65003a11/users/1d66ea357e9c42dd98d3fabf328718c8/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.752344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8dd6-fb46-49db-9dc9-bb29e61f71d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6315ecc51b8a4bc5aaa2b8ec65003a11, user_id=1d66ea357e9c42dd98d3fabf328718c8, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.755782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba8dd6-fb46-49db-9dc9-bb29e61f71d0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.772704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1132/2249] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:22 2022] PUT /identity/v3/projects/6315ecc51b8a4bc5aaa2b8ec65003a11/users/1d66ea357e9c42dd98d3fabf328718c8/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:22.787586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92ac8d-b2c4-4b35-9d26-9cb5a2332184 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:22.788200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92ac8d-b2c4-4b35-9d26-9cb5a2332184 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:22.788641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92ac8d-b2c4-4b35-9d26-9cb5a2332184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:22.797684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:22.799357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:22.812210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1118/2250] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:22.841672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1a92ac8d-b2c4-4b35-9d26-9cb5a2332184 None None] MFA Rules not processed for user `1d66ea357e9c42dd98d3fabf328718c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:22.843549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1a92ac8d-b2c4-4b35-9d26-9cb5a2332184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:22.976025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1133/2251] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:22.988671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:22.990670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:23.006164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8dbdd-4b32-4d22-b6de-307fd1ffb299 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:23.040541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:23.079731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e910ba-d1ce-455a-8351-7364e7c4e644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:23.080635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e910ba-d1ce-455a-8351-7364e7c4e644 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:23.080977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e910ba-d1ce-455a-8351-7364e7c4e644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:23.081252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e910ba-d1ce-455a-8351-7364e7c4e644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:23.094606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e910ba-d1ce-455a-8351-7364e7c4e644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:23.095579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e910ba-d1ce-455a-8351-7364e7c4e644 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:23.121949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1119/2252] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:23.236369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92ac8d-b2c4-4b35-9d26-9cb5a2332184 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:23.256915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92ac8d-b2c4-4b35-9d26-9cb5a2332184 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:23.264901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25d270-a198-403e-8f2d-c13f46f31f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:23.265953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25d270-a198-403e-8f2d-c13f46f31f5f service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:23.266416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25d270-a198-403e-8f2d-c13f46f31f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:23.266898 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25d270-a198-403e-8f2d-c13f46f31f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:23.423626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25d270-a198-403e-8f2d-c13f46f31f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:23.427430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25d270-a198-403e-8f2d-c13f46f31f5f service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:23.450710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1134/2253] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:24.108959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e910ba-d1ce-455a-8351-7364e7c4e644 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:24.214198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e910ba-d1ce-455a-8351-7364e7c4e644 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:24.220717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:24.221619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:24.222120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:24.222530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] PATH_INFO: `/v3/users/9b04ae2d0e8047d7a236422a1502f86c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:24.224030 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b04ae2d0e8047d7a236422a1502f86c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:24.226685 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:24.289297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b04ae2d0e8047d7a236422a1502f86c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:24.290166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b04ae2d0e8047d7a236422a1502f86c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:24.290781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b04ae2d0e8047d7a236422a1502f86c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:24.300212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b04ae2d0e8047d7a236422a1502f86c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:24.317823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b04ae2d0e8047d7a236422a1502f86c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:24.338012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1120/2254] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:24 2022] DELETE /identity/v3/users/9b04ae2d0e8047d7a236422a1502f86c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:24.341533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9372d7-2af3-470a-95ac-30a62e0d7378 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:24.342321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9372d7-2af3-470a-95ac-30a62e0d7378 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:24.342478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9372d7-2af3-470a-95ac-30a62e0d7378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:24.380354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bb9372d7-2af3-470a-95ac-30a62e0d7378 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:24.380866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bb9372d7-2af3-470a-95ac-30a62e0d7378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:24.442380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:24.492889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1135/2255] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:24.507650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:24.510000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:24.669417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18536418-2893-4da5-89d0-f4db22f72946 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:24.677162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:24.677162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:24.678428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:24.678588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:24.697823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:24.702478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:24.705745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:24.731492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1121/2256] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:24.821273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9372d7-2af3-470a-95ac-30a62e0d7378 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:24.824764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bb9372d7-2af3-470a-95ac-30a62e0d7378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:25.002211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9372d7-2af3-470a-95ac-30a62e0d7378 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:25.015373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:25.016747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:25.017724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:25.018999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:25.033669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:25.178479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:25.182369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:25.210901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1136/2257] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:25.286552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:25.289774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:25.322631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed32d1-dfc4-4234-ae14-9294c710773f service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:25.329938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:25.331115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:25.331548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:25.331885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:25.336121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:25.357798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:25.360371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:25.424668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97177d91-1e51-4f48-bf0e-00258e5c0270 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:25.439083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:25.442640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:25.443255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:25.443554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] PATH_INFO: `/v3/projects/9c9e6933eeab476497bd725daff9be90` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:25.446579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9e6933eeab476497bd725daff9be90)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:25.455800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:25.478256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:25.480485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:25.493097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1122/2258] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:25.590807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9e6933eeab476497bd725daff9be90', 'request_id': 'req-03eb1bd5-dac2-4327-817f-bd874d021d2f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:25.599527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9e6933eeab476497bd725daff9be90', 'request_id': 'req-03eb1bd5-dac2-4327-817f-bd874d021d2f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:25.611667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9e6933eeab476497bd725daff9be90', 'request_id': 'req-03eb1bd5-dac2-4327-817f-bd874d021d2f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:25.635672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] The token cache is being invalidate because project 9c9e6933eeab476497bd725daff9be90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:25.636697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:25.637637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:25.639577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1137/2259] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:25 2022] DELETE /identity/v3/projects/9c9e6933eeab476497bd725daff9be90 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:25.646164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:25.672579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa9894c-a27a-4ac1-9a74-3abe1437319f service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:25.838393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:25.839698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:25.840255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:25.840648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] PATH_INFO: `/v3/users/1d66ea357e9c42dd98d3fabf328718c8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:25.845666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d66ea357e9c42dd98d3fabf328718c8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:25.856299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:25.992766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d66ea357e9c42dd98d3fabf328718c8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:25.995254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d66ea357e9c42dd98d3fabf328718c8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:25.997395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d66ea357e9c42dd98d3fabf328718c8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:26.018899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d66ea357e9c42dd98d3fabf328718c8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:26.055431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d66ea357e9c42dd98d3fabf328718c8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:26.090537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1123/2260] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:25 2022] DELETE /identity/v3/users/1d66ea357e9c42dd98d3fabf328718c8 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:26.602043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:26.606602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:26.639941 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:26.700414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb1bd5-dac2-4327-817f-bd874d021d2f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:26.771765 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:26.772290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b91b80-327f-4c37-b096-53ff6d93e719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:26.772937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b91b80-327f-4c37-b096-53ff6d93e719 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:26.773228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b91b80-327f-4c37-b096-53ff6d93e719 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:26.773573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b91b80-327f-4c37-b096-53ff6d93e719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:26.775659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:26.775837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-75b91b80-327f-4c37-b096-53ff6d93e719 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:26.820622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:26.871365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b91b80-327f-4c37-b096-53ff6d93e719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:26.873762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b91b80-327f-4c37-b096-53ff6d93e719 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:26.888562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1138/2261] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:26.982717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd22553-a7e0-49f6-ba05-ecad577c44bd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:27.005876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:27.009530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.012004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.013959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] PATH_INFO: `/v3/projects/6315ecc51b8a4bc5aaa2b8ec65003a11` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.018933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6315ecc51b8a4bc5aaa2b8ec65003a11)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:27.021543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:27.156579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6315ecc51b8a4bc5aaa2b8ec65003a11', 'request_id': 'req-7d30b778-6f6a-46f7-ae96-3883428af996'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:27.175499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6315ecc51b8a4bc5aaa2b8ec65003a11', 'request_id': 'req-7d30b778-6f6a-46f7-ae96-3883428af996'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:27.196241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6315ecc51b8a4bc5aaa2b8ec65003a11', 'request_id': 'req-7d30b778-6f6a-46f7-ae96-3883428af996'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:27.230541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] The token cache is being invalidate because project 6315ecc51b8a4bc5aaa2b8ec65003a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:27.231340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:27.232773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7d30b778-6f6a-46f7-ae96-3883428af996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:27.239812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1124/2262] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:26 2022] DELETE /identity/v3/projects/6315ecc51b8a4bc5aaa2b8ec65003a11 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:27.242124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da303c-1cfb-4717-8293-bfd26273242b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.242418 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da303c-1cfb-4717-8293-bfd26273242b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.242703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da303c-1cfb-4717-8293-bfd26273242b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.279565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b5da303c-1cfb-4717-8293-bfd26273242b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:27.281345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b5da303c-1cfb-4717-8293-bfd26273242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:27.395134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1139/2263] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:27.397495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2ab76-3a4c-4d90-85e1-400792fd381f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.397796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2ab76-3a4c-4d90-85e1-400792fd381f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.398140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2ab76-3a4c-4d90-85e1-400792fd381f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.429227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-06b2ab76-3a4c-4d90-85e1-400792fd381f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:27.430606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-06b2ab76-3a4c-4d90-85e1-400792fd381f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:27.443098 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1125/2264] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:27.444342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da303c-1cfb-4717-8293-bfd26273242b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:27.467344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da303c-1cfb-4717-8293-bfd26273242b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:27.474555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:27.476127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.476435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.476723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.477429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:27.477768 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:27.481547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:27.500110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1140/2265] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:27.505766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2ab76-3a4c-4d90-85e1-400792fd381f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:27.519534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2ab76-3a4c-4d90-85e1-400792fd381f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:27.526043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71073e94-cbff-4bb4-9b05-b5eb0bee8aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:27.527921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073e94-cbff-4bb4-9b05-b5eb0bee8aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.528248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073e94-cbff-4bb4-9b05-b5eb0bee8aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.528724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71073e94-cbff-4bb4-9b05-b5eb0bee8aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.529573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71073e94-cbff-4bb4-9b05-b5eb0bee8aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:27.536790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71073e94-cbff-4bb4-9b05-b5eb0bee8aa1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:27.585938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1126/2266] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:27 2022] POST /identity/v3/projects => generated 359 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:27.590541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:27.605116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58d3ff-f8ec-4bcf-bc26-7c3db63505d5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:27.616922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:27.689798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603c05d-31ba-4aa2-a5ba-4b19726f476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:27.690699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603c05d-31ba-4aa2-a5ba-4b19726f476c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.691241 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603c05d-31ba-4aa2-a5ba-4b19726f476c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.691691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603c05d-31ba-4aa2-a5ba-4b19726f476c admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.692597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603c05d-31ba-4aa2-a5ba-4b19726f476c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:27.695738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603c05d-31ba-4aa2-a5ba-4b19726f476c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:27.731913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1141/2267] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:27 2022] POST /identity/v3/users => generated 280 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:27.736085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71073e94-cbff-4bb4-9b05-b5eb0bee8aa1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:27.761049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71073e94-cbff-4bb4-9b05-b5eb0bee8aa1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:27.770019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8d1f0-253f-4370-8340-7972fe86a04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:27.771035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d1f0-253f-4370-8340-7972fe86a04c admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.771348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d1f0-253f-4370-8340-7972fe86a04c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.771601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8d1f0-253f-4370-8340-7972fe86a04c admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.772324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8d1f0-253f-4370-8340-7972fe86a04c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:27.774890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8d1f0-253f-4370-8340-7972fe86a04c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:27.800295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1127/2268] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:27.801662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603c05d-31ba-4aa2-a5ba-4b19726f476c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:27.824429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603c05d-31ba-4aa2-a5ba-4b19726f476c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:27.831560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caf96c-71ba-47eb-91c7-22b0bbef0350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:27.832718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caf96c-71ba-47eb-91c7-22b0bbef0350 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.837879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caf96c-71ba-47eb-91c7-22b0bbef0350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.837879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caf96c-71ba-47eb-91c7-22b0bbef0350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.837879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caf96c-71ba-47eb-91c7-22b0bbef0350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:27.838459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caf96c-71ba-47eb-91c7-22b0bbef0350 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:27.854447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1142/2269] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:27.860383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8d1f0-253f-4370-8340-7972fe86a04c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:27.879368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8d1f0-253f-4370-8340-7972fe86a04c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:27.900078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:27.900933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.901290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.902781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] PATH_INFO: `/v3/projects/667351f9cf734c2ba8a8bf1d2c294de0/users/6785aec4b0f54f8f89c6176a4c7cb43e/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.923998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=667351f9cf734c2ba8a8bf1d2c294de0, user_id=6785aec4b0f54f8f89c6176a4c7cb43e, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:27.927141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:27.943845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1128/2270] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:27 2022] PUT /identity/v3/projects/667351f9cf734c2ba8a8bf1d2c294de0/users/6785aec4b0f54f8f89c6176a4c7cb43e/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:27.957009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008f8b-9553-4313-98cf-bb1305bb471e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:27.957362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008f8b-9553-4313-98cf-bb1305bb471e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:27.957711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee008f8b-9553-4313-98cf-bb1305bb471e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:27.995186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ee008f8b-9553-4313-98cf-bb1305bb471e None None] MFA Rules not processed for user `6785aec4b0f54f8f89c6176a4c7cb43e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:27.997573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ee008f8b-9553-4313-98cf-bb1305bb471e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:28.108784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1143/2271] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:28.264748 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:28.267909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:28.394009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:28.426431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b97dd-da38-4118-a477-dc996075b0d3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:28.434145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda75e5-a1b7-47e5-9990-01bbb8efecf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:28.435373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda75e5-a1b7-47e5-9990-01bbb8efecf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:28.436266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda75e5-a1b7-47e5-9990-01bbb8efecf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:28.436388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda75e5-a1b7-47e5-9990-01bbb8efecf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:28.453840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda75e5-a1b7-47e5-9990-01bbb8efecf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:28.456682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda75e5-a1b7-47e5-9990-01bbb8efecf8 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:28.476006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1129/2272] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:28.496796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008f8b-9553-4313-98cf-bb1305bb471e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:28.719349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee008f8b-9553-4313-98cf-bb1305bb471e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:28.735434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:28.736127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:28.736399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:28.736680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] PATH_INFO: `/v3/users/6785aec4b0f54f8f89c6176a4c7cb43e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:28.739020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6785aec4b0f54f8f89c6176a4c7cb43e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:28.742912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:28.797405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:28.874664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6785aec4b0f54f8f89c6176a4c7cb43e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:28.931007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6785aec4b0f54f8f89c6176a4c7cb43e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:28.934617 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6785aec4b0f54f8f89c6176a4c7cb43e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:28.964281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6785aec4b0f54f8f89c6176a4c7cb43e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:28.999275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6785aec4b0f54f8f89c6176a4c7cb43e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:29.020269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1144/2273] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:28 2022] DELETE /identity/v3/users/6785aec4b0f54f8f89c6176a4c7cb43e => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:29.020919 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e3ca-9d8f-4368-97e1-a03fc27d2033 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:29.021928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e3ca-9d8f-4368-97e1-a03fc27d2033 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:29.022288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e3ca-9d8f-4368-97e1-a03fc27d2033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:29.075082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b279e3ca-9d8f-4368-97e1-a03fc27d2033 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:29.077985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b279e3ca-9d8f-4368-97e1-a03fc27d2033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:29.360692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1130/2274] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:29.371633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:29.374008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:29.506551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba8eeb-16bc-45c3-969b-9cf65d8a8559 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:29.507402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e3ca-9d8f-4368-97e1-a03fc27d2033 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:29.509754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b279e3ca-9d8f-4368-97e1-a03fc27d2033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:29.510874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4915064-b85b-4a76-bccb-c2b7c4c0d0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:29.512652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915064-b85b-4a76-bccb-c2b7c4c0d0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:29.512940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915064-b85b-4a76-bccb-c2b7c4c0d0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:29.513702 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4915064-b85b-4a76-bccb-c2b7c4c0d0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:29.526009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4915064-b85b-4a76-bccb-c2b7c4c0d0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:29.528176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4915064-b85b-4a76-bccb-c2b7c4c0d0d9 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:29.539138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e3ca-9d8f-4368-97e1-a03fc27d2033 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:29.541089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1145/2275] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:29.548414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:29.549417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:29.556553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:29.556553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] PATH_INFO: `/v3/users/84d886f53b0d44a7910847bb8970a412` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:29.556553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d886f53b0d44a7910847bb8970a412)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:29.556553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:29.684784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d886f53b0d44a7910847bb8970a412'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:29.685496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d886f53b0d44a7910847bb8970a412'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:29.686215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d886f53b0d44a7910847bb8970a412'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:29.696130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d886f53b0d44a7910847bb8970a412'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:29.724006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d886f53b0d44a7910847bb8970a412'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:29.751958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1131/2276] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:29 2022] DELETE /identity/v3/users/84d886f53b0d44a7910847bb8970a412 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:29.757046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faebf4-1713-4f94-b467-851ff755b79b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:29.757939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faebf4-1713-4f94-b467-851ff755b79b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:29.758366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faebf4-1713-4f94-b467-851ff755b79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:29.788815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-45faebf4-1713-4f94-b467-851ff755b79b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:29.790601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-45faebf4-1713-4f94-b467-851ff755b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:29.889445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:29.892036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:29.931312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1146/2277] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:29.945077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45faebf4-1713-4f94-b467-851ff755b79b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:29.946941 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-45faebf4-1713-4f94-b467-851ff755b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:30.026943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:30.029100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:30.056820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45faebf4-1713-4f94-b467-851ff755b79b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:30.069595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:30.070640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.070977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.071369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.075181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fab441-8772-4af6-a737-ddb1b1eafce5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:30.084885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:30.085297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:30.085586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.086029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.086477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] PATH_INFO: `/v3/users/b8defdf430274affaed766d8aedccfad` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.088095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8defdf430274affaed766d8aedccfad)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:30.088303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:30.091607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:30.114029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1147/2278] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:30.140820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:30.142493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:30.175889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51a2e1-9544-4dc8-8140-776300b7dafa service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:30.185318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:30.186619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.187528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.188044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] PATH_INFO: `/v3/projects/667351f9cf734c2ba8a8bf1d2c294de0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.192324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=667351f9cf734c2ba8a8bf1d2c294de0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:30.195779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:30.244431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8defdf430274affaed766d8aedccfad'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.245576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8defdf430274affaed766d8aedccfad'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.246766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8defdf430274affaed766d8aedccfad'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.274699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8defdf430274affaed766d8aedccfad'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.309545 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-3d95373a-9f00-42f1-9033-eb9bd7e3c0ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8defdf430274affaed766d8aedccfad'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.324277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '667351f9cf734c2ba8a8bf1d2c294de0', 'request_id': 'req-4715d34d-a375-414b-aae7-b6dea24f3fbb'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.342443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1132/2279] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:29 2022] DELETE /identity/v3/users/b8defdf430274affaed766d8aedccfad => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:30.351724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '667351f9cf734c2ba8a8bf1d2c294de0', 'request_id': 'req-4715d34d-a375-414b-aae7-b6dea24f3fbb'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.363185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '667351f9cf734c2ba8a8bf1d2c294de0', 'request_id': 'req-4715d34d-a375-414b-aae7-b6dea24f3fbb'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.399743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] The token cache is being invalidate because project 667351f9cf734c2ba8a8bf1d2c294de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:30.399743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.399743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4715d34d-a375-414b-aae7-b6dea24f3fbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.401136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1148/2280] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:30 2022] DELETE /identity/v3/projects/667351f9cf734c2ba8a8bf1d2c294de0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:30.407647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebea473-f0d8-4850-b497-791e14570810 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.407945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebea473-f0d8-4850-b497-791e14570810 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.408229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebea473-f0d8-4850-b497-791e14570810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.437987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-aebea473-f0d8-4850-b497-791e14570810 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:30.439149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-aebea473-f0d8-4850-b497-791e14570810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:30.588015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1133/2281] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:30.594583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de58a6-2592-4e81-b798-41ab768ad3d9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.594886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de58a6-2592-4e81-b798-41ab768ad3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.595191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de58a6-2592-4e81-b798-41ab768ad3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.616245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c4de58a6-2592-4e81-b798-41ab768ad3d9 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:30.617532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c4de58a6-2592-4e81-b798-41ab768ad3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:30.627041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1149/2282] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:30.632780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebea473-f0d8-4850-b497-791e14570810 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:30.644265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebea473-f0d8-4850-b497-791e14570810 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:30.649358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:30.650024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.650308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.650583 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.651261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:30.652130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:30.654319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:30.672378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1134/2283] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:30.673727 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de58a6-2592-4e81-b798-41ab768ad3d9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:30.685751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de58a6-2592-4e81-b798-41ab768ad3d9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:30.693752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea2ba7-57e5-444a-a704-286f22a05f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:30.695579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2ba7-57e5-444a-a704-286f22a05f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.696106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2ba7-57e5-444a-a704-286f22a05f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.696678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea2ba7-57e5-444a-a704-286f22a05f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.698623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea2ba7-57e5-444a-a704-286f22a05f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:30.705166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea2ba7-57e5-444a-a704-286f22a05f3f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:30.746510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:30.749373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:30.753379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1150/2284] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:30 2022] POST /identity/v3/projects => generated 389 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:30.761678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea2ba7-57e5-444a-a704-286f22a05f3f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:30.780956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c18caa-511b-4d73-8bb6-68cacbfc997a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:30.897160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:30.898424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.898839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.899287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] PATH_INFO: `/v3/projects/f75eb0c9b95443eea181e5ee642bee65` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.901031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f75eb0c9b95443eea181e5ee642bee65)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:30.905078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:30.907571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:30.925360 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea2ba7-57e5-444a-a704-286f22a05f3f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:30.933113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:30.934194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:30.934750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:30.935324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:30.936427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:30.939316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:30.968236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1151/2285] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:30 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:30.969442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:30.972507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:30.991699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f75eb0c9b95443eea181e5ee642bee65', 'request_id': 'req-82cc32c6-32be-4b38-9490-212073652eb8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:30.996776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f4424-e923-408f-9638-099ee6a1773d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:31.000083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f75eb0c9b95443eea181e5ee642bee65', 'request_id': 'req-82cc32c6-32be-4b38-9490-212073652eb8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.007289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f75eb0c9b95443eea181e5ee642bee65', 'request_id': 'req-82cc32c6-32be-4b38-9490-212073652eb8'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.024245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] The token cache is being invalidate because project f75eb0c9b95443eea181e5ee642bee65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:31.024672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.025398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.032627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1135/2286] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:30 2022] DELETE /identity/v3/projects/f75eb0c9b95443eea181e5ee642bee65 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:31.032627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:31.032627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:31.035136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:31.075738 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:31.076410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.076693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.076972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] PATH_INFO: `/v3/projects/5f908aee304e4fe2bb2185d89400533c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.078365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f908aee304e4fe2bb2185d89400533c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:31.080268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:31.109593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc32c6-32be-4b38-9490-212073652eb8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:31.114939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:31.115947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.116603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.117060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.118409 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:31.121659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:31.137645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1136/2287] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:31.137645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:31.137950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:31.146946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f908aee304e4fe2bb2185d89400533c', 'request_id': 'req-facdd511-feaa-43ac-94cd-6a8ff93cc151'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.151691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99169ba-3193-45c0-a979-5f203581cdcd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:31.153835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f908aee304e4fe2bb2185d89400533c', 'request_id': 'req-facdd511-feaa-43ac-94cd-6a8ff93cc151'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.164517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f908aee304e4fe2bb2185d89400533c', 'request_id': 'req-facdd511-feaa-43ac-94cd-6a8ff93cc151'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.178902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] The token cache is being invalidate because project 5f908aee304e4fe2bb2185d89400533c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:31.179184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.179912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.181511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1152/2288] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:30 2022] DELETE /identity/v3/projects/5f908aee304e4fe2bb2185d89400533c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:31.182509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:31.184897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:31.229415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:31.230740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.231072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.231368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] PATH_INFO: `/v3/users/c6c83e74fc3349e5bc07a19fef3f6684` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.232801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c83e74fc3349e5bc07a19fef3f6684)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:31.235012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:31.273001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:31.300872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c83e74fc3349e5bc07a19fef3f6684'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.301631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c83e74fc3349e5bc07a19fef3f6684'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.302281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c83e74fc3349e5bc07a19fef3f6684'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.311995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c83e74fc3349e5bc07a19fef3f6684'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.329538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-35c76804-a45e-41c2-b79b-bee124c7cd94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c83e74fc3349e5bc07a19fef3f6684'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:31.361694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1137/2289] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:31 2022] DELETE /identity/v3/users/c6c83e74fc3349e5bc07a19fef3f6684 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:31.363833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426699f-7a95-495d-b9c6-7f16711805cf None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.364331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426699f-7a95-495d-b9c6-7f16711805cf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.364802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426699f-7a95-495d-b9c6-7f16711805cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.400165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0426699f-7a95-495d-b9c6-7f16711805cf None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:31.403286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0426699f-7a95-495d-b9c6-7f16711805cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:31.414972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdd511-feaa-43ac-94cd-6a8ff93cc151 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:31.532249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:31.533486 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.534059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.534671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.536264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1138/2290] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:31.538287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:31.541027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:31.547762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1147d1-6221-4507-9282-5891f24a18d0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.548142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1147d1-6221-4507-9282-5891f24a18d0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.548549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1147d1-6221-4507-9282-5891f24a18d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.577514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1153/2291] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:31.582314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:31.592653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:31.604433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:31.623248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1e1147d1-6221-4507-9282-5891f24a18d0 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:31.624651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1e1147d1-6221-4507-9282-5891f24a18d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:31.626182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef97db-b016-4ba1-8ae4-95f7bb65124f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:31.634727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1139/2292] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:31.635113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb812e-f543-44b8-a16e-4f53ebe02a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:31.639351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb812e-f543-44b8-a16e-4f53ebe02a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.639963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb812e-f543-44b8-a16e-4f53ebe02a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.640095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1147d1-6221-4507-9282-5891f24a18d0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:31.640513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb812e-f543-44b8-a16e-4f53ebe02a76 admin admin] PATH_INFO: `/v3/projects/953aebf5dfa24cdb98618ca5cc54cad4/users/25ed838ac5e044f586f94b2aa5bf0c35/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.650724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1147d1-6221-4507-9282-5891f24a18d0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:31.657878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:31.658616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.658971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.659323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.660074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:31.660438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:31.662581 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:31.662819 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb812e-f543-44b8-a16e-4f53ebe02a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953aebf5dfa24cdb98618ca5cc54cad4, user_id=25ed838ac5e044f586f94b2aa5bf0c35, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:31.668592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb812e-f543-44b8-a16e-4f53ebe02a76 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:31.689954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1140/2294] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:32:31.696981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1154/2294] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:31 2022] PUT /identity/v3/projects/953aebf5dfa24cdb98618ca5cc54cad4/users/25ed838ac5e044f586f94b2aa5bf0c35/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:31.698014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:31.699297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633299-f108-42cb-9c53-bcc6825189c3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.699692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633299-f108-42cb-9c53-bcc6825189c3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.700177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633299-f108-42cb-9c53-bcc6825189c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.720362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5a347-0612-4a89-9906-eee5b1dec16e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:31.742535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-02633299-f108-42cb-9c53-bcc6825189c3 None None] MFA Rules not processed for user `25ed838ac5e044f586f94b2aa5bf0c35`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:31.744495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-02633299-f108-42cb-9c53-bcc6825189c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:31.812982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:31.814404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:31.814785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:31.815139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:31.816344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:31.819064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:31.880407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1141/2295] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:31 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:31.882143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:31.886455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:31.893646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1155/2296] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:31.893646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633299-f108-42cb-9c53-bcc6825189c3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:31.918269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:31.925613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633299-f108-42cb-9c53-bcc6825189c3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.006574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.007198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.007576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.007902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.008612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:32.011041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:32.019252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1341a-9f6a-46ca-b0af-7f88ebda0bbc admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.028807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.029957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.030471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.030941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] PATH_INFO: `/v3/projects/15664109288243539515b798bf33b977` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.033068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15664109288243539515b798bf33b977)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:32.036915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:32.038343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1156/2297] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:31 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:32.039797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:32.043177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:32.136920 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15664109288243539515b798bf33b977', 'request_id': 'req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.142267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce63f55-e1e9-4299-be07-e497761e686b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.150916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15664109288243539515b798bf33b977', 'request_id': 'req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.159897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:32.164279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15664109288243539515b798bf33b977', 'request_id': 'req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.191614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] The token cache is being invalidate because project 15664109288243539515b798bf33b977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:32.191922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.193529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.199874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1142/2298] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:31 2022] DELETE /identity/v3/projects/15664109288243539515b798bf33b977 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:32.199874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:32.199874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:32.243225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.244297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.244798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.247593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.249922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:32.305263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb5f0b-425d-499a-97d3-8c21b17ebe73 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.312063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.312979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.313363 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.313849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.314799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:32.317179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:32.337236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:32.343834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:32.344149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1143/2299] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:32.344149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:32.346597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:32.351319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1157/2300] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:32.352146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:32.366092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b16c-f153-44d6-a25d-4f6a0529e96c service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.372387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d9e7b-61d2-4803-886f-640cbaa23480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.372690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e33f9-8f2f-4223-b813-fc7522a34398 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.373083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9e7b-61d2-4803-886f-640cbaa23480 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.373365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9e7b-61d2-4803-886f-640cbaa23480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.373732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9e7b-61d2-4803-886f-640cbaa23480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.374486 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d9e7b-61d2-4803-886f-640cbaa23480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:32.376355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d9e7b-61d2-4803-886f-640cbaa23480 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:32.381004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.382043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.382604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.383047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] PATH_INFO: `/v3/users/b0f234116ffb4119b676feac40049a8f` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.384864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f234116ffb4119b676feac40049a8f)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:32.388503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:32.391981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1158/2301] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:32.396874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d9e7b-61d2-4803-886f-640cbaa23480 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:32.409862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d9e7b-61d2-4803-886f-640cbaa23480 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.419161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cef4b5-b8ad-4ded-ad0a-1a6638220b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.420884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cef4b5-b8ad-4ded-ad0a-1a6638220b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.420884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cef4b5-b8ad-4ded-ad0a-1a6638220b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.421173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cef4b5-b8ad-4ded-ad0a-1a6638220b21 admin admin] PATH_INFO: `/v3/projects/1432ad4a21624574b8caeaf10f2f8339/users/4471a60040ca42649f2718fd46659e31/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.450858 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:32.454403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cef4b5-b8ad-4ded-ad0a-1a6638220b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1432ad4a21624574b8caeaf10f2f8339, user_id=4471a60040ca42649f2718fd46659e31, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:32.460141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cef4b5-b8ad-4ded-ad0a-1a6638220b21 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:32.482310 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f234116ffb4119b676feac40049a8f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.483633 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1159/2302] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:32 2022] PUT /identity/v3/projects/1432ad4a21624574b8caeaf10f2f8339/users/4471a60040ca42649f2718fd46659e31/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:32.483830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f234116ffb4119b676feac40049a8f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.485772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f234116ffb4119b676feac40049a8f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.491800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a338ec-0e7a-4038-8182-9b432f63c453 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.492416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a338ec-0e7a-4038-8182-9b432f63c453 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.492905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a338ec-0e7a-4038-8182-9b432f63c453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.501063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f234116ffb4119b676feac40049a8f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.524048 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f234116ffb4119b676feac40049a8f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:32.532118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c8a338ec-0e7a-4038-8182-9b432f63c453 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:32.535225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c8a338ec-0e7a-4038-8182-9b432f63c453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:32.546464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1144/2303] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:32 2022] DELETE /identity/v3/users/b0f234116ffb4119b676feac40049a8f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:32.670167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1160/2304] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:32.685051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:32.686833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:32.800697 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd913580-4099-4fd9-ac73-5c7cdd7aa3df admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.809118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.810055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.810598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.811069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.815870 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a338ec-0e7a-4038-8182-9b432f63c453 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:32.817976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c8a338ec-0e7a-4038-8182-9b432f63c453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:32.835742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a338ec-0e7a-4038-8182-9b432f63c453 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:32.894983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:32.940304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:32.941141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:32.941554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:32.943298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] PATH_INFO: `/v3/users/7fcee04e4b2e478fa47b77de20f6658b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:32.943298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fcee04e4b2e478fa47b77de20f6658b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:32.946255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:33.050529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:33.058290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:33.068878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fcee04e4b2e478fa47b77de20f6658b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.081880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1145/2305] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:33.082775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:33.084215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:33.098397 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fcee04e4b2e478fa47b77de20f6658b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.100164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fcee04e4b2e478fa47b77de20f6658b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.100747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678cf78-447c-4748-a21d-6dcd83847e29 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:33.110118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:33.111059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:33.111373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:33.111870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] PATH_INFO: `/v3/projects/d7d67da618e048408da1c1a290da92bf` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:33.114911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fcee04e4b2e478fa47b77de20f6658b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.122017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d67da618e048408da1c1a290da92bf)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:33.124699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:33.133072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fa5558-9012-4a23-9fe7-04a39da0bb10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fcee04e4b2e478fa47b77de20f6658b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.151389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1161/2306] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:32 2022] DELETE /identity/v3/users/7fcee04e4b2e478fa47b77de20f6658b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:33.159166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473b80b-0f06-4d3b-bdbb-4f9c7f0265f5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:33.159951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473b80b-0f06-4d3b-bdbb-4f9c7f0265f5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:33.160814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473b80b-0f06-4d3b-bdbb-4f9c7f0265f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:33.231593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c473b80b-0f06-4d3b-bdbb-4f9c7f0265f5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:33.232513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d67da618e048408da1c1a290da92bf', 'request_id': 'req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.232865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c473b80b-0f06-4d3b-bdbb-4f9c7f0265f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:33.250006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d67da618e048408da1c1a290da92bf', 'request_id': 'req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.252446 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:33.263458 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d67da618e048408da1c1a290da92bf', 'request_id': 'req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.298429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] The token cache is being invalidate because project d7d67da618e048408da1c1a290da92bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:33.299353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.301066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-1d1340ab-ad10-45e4-9aaf-7dbcf4c2a09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:33.306271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1146/2307] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:33 2022] DELETE /identity/v3/projects/d7d67da618e048408da1c1a290da92bf => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:33.315547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a663-feec-4b89-a13c-b204e9410c56 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:33.315889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a663-feec-4b89-a13c-b204e9410c56 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:33.317172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930a663-feec-4b89-a13c-b204e9410c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:33.352450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8930a663-feec-4b89-a13c-b204e9410c56 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:33.353501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1162/2308] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:33.355053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8930a663-feec-4b89-a13c-b204e9410c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:33.365106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473b80b-0f06-4d3b-bdbb-4f9c7f0265f5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:33.367627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c473b80b-0f06-4d3b-bdbb-4f9c7f0265f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:33.372930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1147/2309] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:33.390626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992caad-3f1c-46a9-be62-53a031e70768 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:33.391323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992caad-3f1c-46a9-be62-53a031e70768 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:33.391854 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992caad-3f1c-46a9-be62-53a031e70768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:33.431271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1992caad-3f1c-46a9-be62-53a031e70768 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:33.434335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1992caad-3f1c-46a9-be62-53a031e70768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:33.450014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1148/2310] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:33.454924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992caad-3f1c-46a9-be62-53a031e70768 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:33.472549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992caad-3f1c-46a9-be62-53a031e70768 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:33.490966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:33.492503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:33.492991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:33.493559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:33.494398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:33.495965 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:33.498003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:33.514447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473b80b-0f06-4d3b-bdbb-4f9c7f0265f5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:33.518675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1149/2311] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:32:33.520612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464f0d3-970c-41b6-986b-2f1fed825d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:33.521479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464f0d3-970c-41b6-986b-2f1fed825d01 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:33.522064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464f0d3-970c-41b6-986b-2f1fed825d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:33.522512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464f0d3-970c-41b6-986b-2f1fed825d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:33.527595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:33.534681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:33.540381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464f0d3-970c-41b6-986b-2f1fed825d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:33.544605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a3e44-b256-495a-b1c7-e9c2c5f18cdd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:33.546995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464f0d3-970c-41b6-986b-2f1fed825d01 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:33.552481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef20135-f1ca-4d3a-b5d1-cf0df55b6e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:33.553341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20135-f1ca-4d3a-b5d1-cf0df55b6e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:33.553778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20135-f1ca-4d3a-b5d1-cf0df55b6e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:33.554147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef20135-f1ca-4d3a-b5d1-cf0df55b6e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:33.555027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef20135-f1ca-4d3a-b5d1-cf0df55b6e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:33.557380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef20135-f1ca-4d3a-b5d1-cf0df55b6e29 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:33.567910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1163/2312] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:33.603840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1150/2313] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:33 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:33.609937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464f0d3-970c-41b6-986b-2f1fed825d01 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:33.644879 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464f0d3-970c-41b6-986b-2f1fed825d01 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:33.887019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dd6b4-d677-4093-846d-ea186dffb42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:33.889607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dd6b4-d677-4093-846d-ea186dffb42a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:33.890111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dd6b4-d677-4093-846d-ea186dffb42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:33.890674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768dd6b4-d677-4093-846d-ea186dffb42a admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:33.892414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768dd6b4-d677-4093-846d-ea186dffb42a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:33.896820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768dd6b4-d677-4093-846d-ea186dffb42a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:33.963610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1164/2314] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:33 2022] POST /identity/v3/users => generated 289 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:33.970362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef20135-f1ca-4d3a-b5d1-cf0df55b6e29 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:34.005560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef20135-f1ca-4d3a-b5d1-cf0df55b6e29 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:34.024222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f4863-1645-432b-bdba-3990193309bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:34.024872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f4863-1645-432b-bdba-3990193309bf admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:34.025157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f4863-1645-432b-bdba-3990193309bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:34.025439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f4863-1645-432b-bdba-3990193309bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:34.026165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f4863-1645-432b-bdba-3990193309bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:34.031500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f4863-1645-432b-bdba-3990193309bf admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:34.056552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1151/2315] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:34.063312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dd6b4-d677-4093-846d-ea186dffb42a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:34.124906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768dd6b4-d677-4093-846d-ea186dffb42a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:34.132414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:34.133504 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:34.133952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:34.134384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:34.135222 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:34.138388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:34.158143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1165/2316] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:34.158963 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f4863-1645-432b-bdba-3990193309bf admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:34.178944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f4863-1645-432b-bdba-3990193309bf admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:34.188731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c35c9c-5567-457a-8da3-c6709ae52176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:34.189598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c35c9c-5567-457a-8da3-c6709ae52176 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:34.189899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c35c9c-5567-457a-8da3-c6709ae52176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:34.190236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c35c9c-5567-457a-8da3-c6709ae52176 admin admin] PATH_INFO: `/v3/projects/f5a68ad78ec547dcb0036549fbca1d49/users/4eb3df7400404108b9a215fb69f0bf3f/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:34.232453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c35c9c-5567-457a-8da3-c6709ae52176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a68ad78ec547dcb0036549fbca1d49, user_id=4eb3df7400404108b9a215fb69f0bf3f, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:34.235038 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c35c9c-5567-457a-8da3-c6709ae52176 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:34.279463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:34.297496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1152/2317] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:34 2022] PUT /identity/v3/projects/f5a68ad78ec547dcb0036549fbca1d49/users/4eb3df7400404108b9a215fb69f0bf3f/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:34.297993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:34.310279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef123a1-45cd-4248-bd69-ae657251d3df None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:34.310810 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef123a1-45cd-4248-bd69-ae657251d3df None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:34.311297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef123a1-45cd-4248-bd69-ae657251d3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:34.316959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63c6c4-1ad4-4caf-84c4-8f586ed6cdf6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:34.417510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cef123a1-45cd-4248-bd69-ae657251d3df None None] MFA Rules not processed for user `4eb3df7400404108b9a215fb69f0bf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:34.420463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cef123a1-45cd-4248-bd69-ae657251d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:34.456929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:34.460433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:34.460769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:34.460992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] PATH_INFO: `/v3/projects/6e3ee64f20864e4daffcccbd8a7b2be5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:34.465023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3ee64f20864e4daffcccbd8a7b2be5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:34.470253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:34.611280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1153/2318] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:34.612176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3ee64f20864e4daffcccbd8a7b2be5', 'request_id': 'req-b679aa13-ea8a-4941-83d6-3212b5b8300f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.627163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef123a1-45cd-4248-bd69-ae657251d3df None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:34.642371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cef123a1-45cd-4248-bd69-ae657251d3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:34.644013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3ee64f20864e4daffcccbd8a7b2be5', 'request_id': 'req-b679aa13-ea8a-4941-83d6-3212b5b8300f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.658013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3ee64f20864e4daffcccbd8a7b2be5', 'request_id': 'req-b679aa13-ea8a-4941-83d6-3212b5b8300f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.690108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] The token cache is being invalidate because project 6e3ee64f20864e4daffcccbd8a7b2be5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:34.690108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.690108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.692211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1166/2319] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:34 2022] DELETE /identity/v3/projects/6e3ee64f20864e4daffcccbd8a7b2be5 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:34.699908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:34.701372 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:34.715771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:34.775767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:34.799544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679aa13-ea8a-4941-83d6-3212b5b8300f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:34.811212 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:34.812416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:34.812785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:34.813069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] PATH_INFO: `/v3/users/edcd85bc334f41719b0713c80642a55a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:34.815461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=edcd85bc334f41719b0713c80642a55a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:34.817494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:34.830287 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef123a1-45cd-4248-bd69-ae657251d3df None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:34.884914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edcd85bc334f41719b0713c80642a55a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.907187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edcd85bc334f41719b0713c80642a55a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.908954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edcd85bc334f41719b0713c80642a55a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.920803 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edcd85bc334f41719b0713c80642a55a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.934033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edcd85bc334f41719b0713c80642a55a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:34.944454 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1167/2320] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:34 2022] DELETE /identity/v3/users/edcd85bc334f41719b0713c80642a55a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:34.993114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '464190e0ccaf4f82942f518df1bf6c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:34.993573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:34.994177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:34.994837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:35.000172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:35.130376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:35.131868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:35.156255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1154/2321] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:35.540254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:35.543094 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:35.733756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c6397-a7c3-496d-89c5-c7cd708a43eb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:35.788257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:35.789398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:35.789916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:35.794194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] PATH_INFO: `/v3/projects/f94197d784eb4a66999063169e0a4562` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:35.801593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f94197d784eb4a66999063169e0a4562)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:35.821568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:35.886869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:35.961934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f94197d784eb4a66999063169e0a4562', 'request_id': 'req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:35.978563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f94197d784eb4a66999063169e0a4562', 'request_id': 'req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:35.989350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f94197d784eb4a66999063169e0a4562', 'request_id': 'req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:36.010319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] The token cache is being invalidate because project f94197d784eb4a66999063169e0a4562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:36.011062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:36.012059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:36.014329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1168/2322] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:35 2022] DELETE /identity/v3/projects/f94197d784eb4a66999063169e0a4562 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:36.019849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:36.036528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc26d34-a017-4867-af86-58975ba74663 service glance] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:36.152159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:36.153119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:36.153492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:36.153852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] PATH_INFO: `/v3/users/c7a18ca4a3944d21812252a10b66ba42` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:36.155525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a18ca4a3944d21812252a10b66ba42)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:36.160372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:36.221945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:36.230463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a18ca4a3944d21812252a10b66ba42'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:36.231185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a18ca4a3944d21812252a10b66ba42'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:36.231828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a18ca4a3944d21812252a10b66ba42'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:36.264211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a18ca4a3944d21812252a10b66ba42'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:36.289897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d21a8345-1bbb-4db3-a620-c1d1116317f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a18ca4a3944d21812252a10b66ba42'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:36.314780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1155/2323] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:36 2022] DELETE /identity/v3/users/c7a18ca4a3944d21812252a10b66ba42 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:36.914251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:36.915985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:36.939882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcd1cd-c0fc-4dd0-8b76-025ce3ee4412 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:37.050244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:37.051173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:37.051173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:37.051437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] PATH_INFO: `/v3/projects/e905cdccc5c241c09832623a7396868e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:37.052694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e905cdccc5c241c09832623a7396868e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:37.054750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:37.132581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e905cdccc5c241c09832623a7396868e', 'request_id': 'req-7caa6caa-5886-4f06-a442-3f921d523eb6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:37.141307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e905cdccc5c241c09832623a7396868e', 'request_id': 'req-7caa6caa-5886-4f06-a442-3f921d523eb6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:37.157579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e905cdccc5c241c09832623a7396868e', 'request_id': 'req-7caa6caa-5886-4f06-a442-3f921d523eb6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:37.188546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] The token cache is being invalidate because project e905cdccc5c241c09832623a7396868e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:37.189263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:37.190618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7caa6caa-5886-4f06-a442-3f921d523eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:37.196805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1169/2324] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:36 2022] DELETE /identity/v3/projects/e905cdccc5c241c09832623a7396868e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:37.204192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e41f-f612-43da-bf3e-39b5687fe642 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:37.204752 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e41f-f612-43da-bf3e-39b5687fe642 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:37.205307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e41f-f612-43da-bf3e-39b5687fe642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:37.257682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8d79e41f-f612-43da-bf3e-39b5687fe642 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:37.261724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8d79e41f-f612-43da-bf3e-39b5687fe642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:37.370946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1156/2325] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:37.384428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4540e-97b9-4899-b6d1-dc9804418a2b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:37.384947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4540e-97b9-4899-b6d1-dc9804418a2b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:37.385482 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4540e-97b9-4899-b6d1-dc9804418a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:37.424194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a5a4540e-97b9-4899-b6d1-dc9804418a2b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:37.424194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a5a4540e-97b9-4899-b6d1-dc9804418a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:37.454098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1170/2326] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:37.461165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79e41f-f612-43da-bf3e-39b5687fe642 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:37.482676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79e41f-f612-43da-bf3e-39b5687fe642 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:37.496905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:37.510113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:37.511542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:37.511894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:37.513219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:37.515597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:37.518407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:37.538025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1157/2327] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:37.545029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4540e-97b9-4899-b6d1-dc9804418a2b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:37.587391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4540e-97b9-4899-b6d1-dc9804418a2b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:37.597987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec33f8-a6b6-42bb-9ad1-7170ef72b5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:37.599295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec33f8-a6b6-42bb-9ad1-7170ef72b5ca admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:37.599836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec33f8-a6b6-42bb-9ad1-7170ef72b5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:37.600253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec33f8-a6b6-42bb-9ad1-7170ef72b5ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:37.601547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec33f8-a6b6-42bb-9ad1-7170ef72b5ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:37.605509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec33f8-a6b6-42bb-9ad1-7170ef72b5ca admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:37.667538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1171/2328] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:37 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:37.669280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:37.701500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918ad79-4e59-43a5-b253-5e7c83b97803 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:37.814148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cc472-aefb-490a-a241-a8baaf4dd935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:37.814964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cc472-aefb-490a-a241-a8baaf4dd935 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:37.815276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cc472-aefb-490a-a241-a8baaf4dd935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:37.815773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cc472-aefb-490a-a241-a8baaf4dd935 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:37.816675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cc472-aefb-490a-a241-a8baaf4dd935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:37.819111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cc472-aefb-490a-a241-a8baaf4dd935 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:37.856759 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1158/2329] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:37 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:37.868069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec33f8-a6b6-42bb-9ad1-7170ef72b5ca admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:37.887881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec33f8-a6b6-42bb-9ad1-7170ef72b5ca admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:37.894448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4da6-304c-4ed8-9566-cfe37fa47334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:37.895086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4da6-304c-4ed8-9566-cfe37fa47334 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:37.895459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4da6-304c-4ed8-9566-cfe37fa47334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:37.895989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4da6-304c-4ed8-9566-cfe37fa47334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:37.896907 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4da6-304c-4ed8-9566-cfe37fa47334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:37.898706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4da6-304c-4ed8-9566-cfe37fa47334 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:37.912876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1172/2330] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:37.916473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cc472-aefb-490a-a241-a8baaf4dd935 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:37.929224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cc472-aefb-490a-a241-a8baaf4dd935 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:37.936503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198e3cb-4d1b-4f2c-916c-9b8cb6f73048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:37.937326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198e3cb-4d1b-4f2c-916c-9b8cb6f73048 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:37.937711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198e3cb-4d1b-4f2c-916c-9b8cb6f73048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:37.938073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198e3cb-4d1b-4f2c-916c-9b8cb6f73048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:37.938817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198e3cb-4d1b-4f2c-916c-9b8cb6f73048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:37.940688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198e3cb-4d1b-4f2c-916c-9b8cb6f73048 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:37.955005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1159/2331] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:37.960220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4da6-304c-4ed8-9566-cfe37fa47334 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:37.970028 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:37.984154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4da6-304c-4ed8-9566-cfe37fa47334 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:38.004257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:38.009500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:38.010162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:38.010442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] PATH_INFO: `/v3/projects/afa1958afb874a0cba4c57df10f2ce08/users/9b293568e74c4c69baee5cb29013ab6d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:38.058498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa1958afb874a0cba4c57df10f2ce08, user_id=9b293568e74c4c69baee5cb29013ab6d, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:38.066657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:38.106375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1173/2332] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:37 2022] PUT /identity/v3/projects/afa1958afb874a0cba4c57df10f2ce08/users/9b293568e74c4c69baee5cb29013ab6d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:38.114976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73041d5-3b82-4e65-a388-76a5c8607528 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:38.115455 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73041d5-3b82-4e65-a388-76a5c8607528 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:38.116187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73041d5-3b82-4e65-a388-76a5c8607528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:38.155215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e73041d5-3b82-4e65-a388-76a5c8607528 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:38.158533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e73041d5-3b82-4e65-a388-76a5c8607528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:38.218199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:38.336397 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1160/2333] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:38.351592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:38.354915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:38.518495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f192d-b9a0-4e75-a23d-e48a2e6a072b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:38.523663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:38.523663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:38.523949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:38.524216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:38.551560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:38.554549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:38.578480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1174/2334] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:39.993786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c274-dec9-4d5f-80ea-cfb0864fe23f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:39.995330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c274-dec9-4d5f-80ea-cfb0864fe23f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:39.996072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49c274-dec9-4d5f-80ea-cfb0864fe23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:40.058870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-cf49c274-dec9-4d5f-80ea-cfb0864fe23f None None] MFA Rules not processed for user `9b293568e74c4c69baee5cb29013ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:40.061183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cf49c274-dec9-4d5f-80ea-cfb0864fe23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:40.267376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1161/2335] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:40.285119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:40.288507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:40.484459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84cc6b2-b07b-48eb-839e-2451fb9deb93 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:40.495058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:40.496287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:40.496939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:40.497900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:40.518025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:40.521136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:40.559737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1175/2336] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:40.624885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76a234-1851-4ba2-b3ae-106bfec36038 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:40.625162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76a234-1851-4ba2-b3ae-106bfec36038 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:40.625443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76a234-1851-4ba2-b3ae-106bfec36038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:40.669108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ee76a234-1851-4ba2-b3ae-106bfec36038 None None] MFA Rules not processed for user `4471a60040ca42649f2718fd46659e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:40.671871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ee76a234-1851-4ba2-b3ae-106bfec36038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:40.809491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1162/2337] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:40.835190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:40.838715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:40.985331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaddc6d-b48a-460a-bd1c-85262599d5e3 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:40.992973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a41dfa-7f25-4ce3-a861-e8f2ed5688b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:40.993657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41dfa-7f25-4ce3-a861-e8f2ed5688b6 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:40.993950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41dfa-7f25-4ce3-a861-e8f2ed5688b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:40.994396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a41dfa-7f25-4ce3-a861-e8f2ed5688b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:41.005149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a41dfa-7f25-4ce3-a861-e8f2ed5688b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:41.007654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a41dfa-7f25-4ce3-a861-e8f2ed5688b6 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:41.028837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1176/2338] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:43.022346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c0cf-4dac-4587-b45b-08ec1aa78b03 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:43.023198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c0cf-4dac-4587-b45b-08ec1aa78b03 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:43.023198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741c0cf-4dac-4587-b45b-08ec1aa78b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:43.062709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3741c0cf-4dac-4587-b45b-08ec1aa78b03 None None] MFA Rules not processed for user `4471a60040ca42649f2718fd46659e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:43.064170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3741c0cf-4dac-4587-b45b-08ec1aa78b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:43.074444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1163/2339] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:44.479220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6c70-e949-474f-a3f9-23bf7282095f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:44.480025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6c70-e949-474f-a3f9-23bf7282095f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:44.480025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd6c70-e949-474f-a3f9-23bf7282095f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:44.504020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741c0cf-4dac-4587-b45b-08ec1aa78b03 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:44.506463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3741c0cf-4dac-4587-b45b-08ec1aa78b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:44.538315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741c0cf-4dac-4587-b45b-08ec1aa78b03 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:44.539191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d6dd6c70-e949-474f-a3f9-23bf7282095f None None] MFA Rules not processed for user `9b293568e74c4c69baee5cb29013ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:44.540478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d6dd6c70-e949-474f-a3f9-23bf7282095f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:44.554253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1177/2340] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:44.569099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:44.570168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:44.570628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:44.571050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] PATH_INFO: `/v3/users/de09cae8e2204f7c91dc3764040588be` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:44.577094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de09cae8e2204f7c91dc3764040588be)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:44.584687 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:44.737032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:44.812707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de09cae8e2204f7c91dc3764040588be'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:44.814299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de09cae8e2204f7c91dc3764040588be'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:44.820274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de09cae8e2204f7c91dc3764040588be'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:44.859691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de09cae8e2204f7c91dc3764040588be'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:44.885629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de09cae8e2204f7c91dc3764040588be'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:44.914670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1164/2341] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:44 2022] DELETE /identity/v3/users/de09cae8e2204f7c91dc3764040588be => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:44.924546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac40a1-80a5-4bea-b1a8-c775aec1643e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:44.925812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac40a1-80a5-4bea-b1a8-c775aec1643e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:44.927053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac40a1-80a5-4bea-b1a8-c775aec1643e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:44.939734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:44.966042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3bac40a1-80a5-4bea-b1a8-c775aec1643e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:44.983892 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3bac40a1-80a5-4bea-b1a8-c775aec1643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:45.225062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1178/2342] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:45.250234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:45.254343 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:45.398016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ab219-47a8-407e-a899-9318783198d9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:45.412575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f9c58-6712-428b-aa11-c38cc147656e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:45.413829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f9c58-6712-428b-aa11-c38cc147656e service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:45.414659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f9c58-6712-428b-aa11-c38cc147656e service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:45.415247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f9c58-6712-428b-aa11-c38cc147656e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:45.450065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f9c58-6712-428b-aa11-c38cc147656e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:45.452262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f9c58-6712-428b-aa11-c38cc147656e service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:45.480883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1165/2343] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:46.146621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac40a1-80a5-4bea-b1a8-c775aec1643e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:46.148161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3bac40a1-80a5-4bea-b1a8-c775aec1643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:46.172411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac40a1-80a5-4bea-b1a8-c775aec1643e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:46.203473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:46.204244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:46.204478 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:46.204809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] PATH_INFO: `/v3/projects/30be75cdd1f949f1894a5052cbd6755c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:46.206284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] RBAC: Authorizing `identity:delete_project(project_id=30be75cdd1f949f1894a5052cbd6755c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:46.208287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:46.396166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30be75cdd1f949f1894a5052cbd6755c', 'request_id': 'req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:46.408851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30be75cdd1f949f1894a5052cbd6755c', 'request_id': 'req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:46.431162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30be75cdd1f949f1894a5052cbd6755c', 'request_id': 'req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:46.457783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] The token cache is being invalidate because project 30be75cdd1f949f1894a5052cbd6755c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:46.458141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:46.461993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e45ecd5b-01ba-4ebb-ae17-d862ba27f82e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:46.471609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1179/2344] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:46 2022] DELETE /identity/v3/projects/30be75cdd1f949f1894a5052cbd6755c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:46.474905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136882-0df4-4592-b8f4-d6ce986a5dd0 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:46.475523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136882-0df4-4592-b8f4-d6ce986a5dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:46.476484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7136882-0df4-4592-b8f4-d6ce986a5dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:46.523704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c7136882-0df4-4592-b8f4-d6ce986a5dd0 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:46.527725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c7136882-0df4-4592-b8f4-d6ce986a5dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:46.688324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1166/2345] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:46.696138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542952-a654-4d2a-90e1-637576dcd006 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:46.696949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542952-a654-4d2a-90e1-637576dcd006 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:46.697583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15542952-a654-4d2a-90e1-637576dcd006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:46.721881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-15542952-a654-4d2a-90e1-637576dcd006 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:46.723253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-15542952-a654-4d2a-90e1-637576dcd006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:46.736472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1180/2346] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:46.754419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7136882-0df4-4592-b8f4-d6ce986a5dd0 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:46.767475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7136882-0df4-4592-b8f4-d6ce986a5dd0 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:46.776544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:46.777593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:46.777940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:46.778225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:46.779061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:46.779377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:46.781577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:46.803534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1167/2347] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:46.808578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542952-a654-4d2a-90e1-637576dcd006 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:46.820635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15542952-a654-4d2a-90e1-637576dcd006 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:46.827033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe005-c623-449e-a26c-a3f5e46f9f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:46.827680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe005-c623-449e-a26c-a3f5e46f9f66 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:46.827967 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe005-c623-449e-a26c-a3f5e46f9f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:46.828247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe005-c623-449e-a26c-a3f5e46f9f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:46.828918 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebe005-c623-449e-a26c-a3f5e46f9f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:46.830953 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebe005-c623-449e-a26c-a3f5e46f9f66 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:46.882831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1181/2348] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:46 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:46.888207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:46.909907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0748c-2d52-4b89-8243-4d42efa54c36 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:46.981192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:47.020120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbcc3e-0f19-4cb9-aba4-0bda9627abb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:47.020868 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbcc3e-0f19-4cb9-aba4-0bda9627abb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:47.021250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbcc3e-0f19-4cb9-aba4-0bda9627abb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:47.021906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbcc3e-0f19-4cb9-aba4-0bda9627abb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:47.022695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cbcc3e-0f19-4cb9-aba4-0bda9627abb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:47.026068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cbcc3e-0f19-4cb9-aba4-0bda9627abb8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:47.064335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1168/2349] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:46 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:47.066031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe005-c623-449e-a26c-a3f5e46f9f66 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:47.090739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe005-c623-449e-a26c-a3f5e46f9f66 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:47.098012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc207e-c222-4ccc-90af-8c43ae8f9f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:47.100642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc207e-c222-4ccc-90af-8c43ae8f9f9b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:47.101100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc207e-c222-4ccc-90af-8c43ae8f9f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:47.101606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc207e-c222-4ccc-90af-8c43ae8f9f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:47.102799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc207e-c222-4ccc-90af-8c43ae8f9f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:47.105784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc207e-c222-4ccc-90af-8c43ae8f9f9b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:47.131678 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1182/2350] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:47.137424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbcc3e-0f19-4cb9-aba4-0bda9627abb8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:47.172516 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbcc3e-0f19-4cb9-aba4-0bda9627abb8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:47.180957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ee4e2-2162-4fea-9205-7bc80eccf66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:47.181797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ee4e2-2162-4fea-9205-7bc80eccf66b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:47.182825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ee4e2-2162-4fea-9205-7bc80eccf66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:47.183170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ee4e2-2162-4fea-9205-7bc80eccf66b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:47.183978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ee4e2-2162-4fea-9205-7bc80eccf66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:47.186312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ee4e2-2162-4fea-9205-7bc80eccf66b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:47.200628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1169/2351] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:32:47.205749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc207e-c222-4ccc-90af-8c43ae8f9f9b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:47.224671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc207e-c222-4ccc-90af-8c43ae8f9f9b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:47.235069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:47.235987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:47.236489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:47.236847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] PATH_INFO: `/v3/projects/33af9400d270489da18c21dee3f9188f/users/75e5a47c5b314a0486d3a32ffbead793/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:47.255099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=33af9400d270489da18c21dee3f9188f, user_id=75e5a47c5b314a0486d3a32ffbead793, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:47.258044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:47.279991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1183/2352] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:47 2022] PUT /identity/v3/projects/33af9400d270489da18c21dee3f9188f/users/75e5a47c5b314a0486d3a32ffbead793/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:47.288372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab2b0-909f-4a98-8246-ba962e581099 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:47.289080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab2b0-909f-4a98-8246-ba962e581099 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:47.289898 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ab2b0-909f-4a98-8246-ba962e581099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:47.313087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a17ab2b0-909f-4a98-8246-ba962e581099 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:47.314513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a17ab2b0-909f-4a98-8246-ba962e581099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:47.422589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1170/2353] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:47.434613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:47.436681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:47.487800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:47.567840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0d8e4-fcf8-4ff1-8b6e-aa5d822ce47d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:47.578189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:47.578828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:47.579107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:47.579387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:47.594534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:47.597047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:47.620073 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1184/2354] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:49.099431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9cc10-125b-4414-9fb6-74a8832f8430 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:49.100540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9cc10-125b-4414-9fb6-74a8832f8430 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:49.100995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9cc10-125b-4414-9fb6-74a8832f8430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:49.152165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5aa9cc10-125b-4414-9fb6-74a8832f8430 None None] MFA Rules not processed for user `75e5a47c5b314a0486d3a32ffbead793`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:49.153739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5aa9cc10-125b-4414-9fb6-74a8832f8430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:49.280612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1171/2355] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:49.293134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:49.297273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:49.429107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ae13a-d1af-4cd1-8694-a4537e9da6f8 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:49.434382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:49.435249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:49.435675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:49.436018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:49.457865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:49.459990 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:49.473560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1185/2356] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:32:51.212045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370c70-1a6d-45ca-938f-9058790a3e75 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:51.213061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370c70-1a6d-45ca-938f-9058790a3e75 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:51.213676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370c70-1a6d-45ca-938f-9058790a3e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:51.238807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-64370c70-1a6d-45ca-938f-9058790a3e75 None None] MFA Rules not processed for user `75e5a47c5b314a0486d3a32ffbead793`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:51.240449 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-64370c70-1a6d-45ca-938f-9058790a3e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:51.248385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1172/2357] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:54.860443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:54.862893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:54.890550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd8fc5-7da1-4901-b3bc-eb3df7a2ab42 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:54.898563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:54.899649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:54.900148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:54.901185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] PATH_INFO: `/v3/users/25ed838ac5e044f586f94b2aa5bf0c35` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:54.903954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ed838ac5e044f586f94b2aa5bf0c35)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:54.908029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:55.018508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ed838ac5e044f586f94b2aa5bf0c35'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.050320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ed838ac5e044f586f94b2aa5bf0c35'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.051593 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ed838ac5e044f586f94b2aa5bf0c35'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.077745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ed838ac5e044f586f94b2aa5bf0c35'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.093573 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ed838ac5e044f586f94b2aa5bf0c35'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.106560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1186/2358] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:32:54 2022] DELETE /identity/v3/users/25ed838ac5e044f586f94b2aa5bf0c35 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:32:55.113692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e704d-aa2b-4168-aa40-b72be61e73c8 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:55.114239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e704d-aa2b-4168-aa40-b72be61e73c8 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:55.114393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e704d-aa2b-4168-aa40-b72be61e73c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:55.147940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-717e704d-aa2b-4168-aa40-b72be61e73c8 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:55.151278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-717e704d-aa2b-4168-aa40-b72be61e73c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:55.214071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:55.257735 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1173/2359] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:55.271582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:55.273824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:55.382978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:55.400683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894bcba-ca75-44d8-b253-54f5a27df797 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:55.406100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab98527-191b-4c1f-930f-2daf06f55f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:55.406737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab98527-191b-4c1f-930f-2daf06f55f70 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:55.407149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab98527-191b-4c1f-930f-2daf06f55f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:55.408041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab98527-191b-4c1f-930f-2daf06f55f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:55.420556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab98527-191b-4c1f-930f-2daf06f55f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:55.422619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab98527-191b-4c1f-930f-2daf06f55f70 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:55.436408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1187/2360] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:32:55.815917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e704d-aa2b-4168-aa40-b72be61e73c8 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:55.817883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-717e704d-aa2b-4168-aa40-b72be61e73c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:55.837628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e704d-aa2b-4168-aa40-b72be61e73c8 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:55.843065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:55.843738 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:55.844016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:55.844495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] PATH_INFO: `/v3/projects/953aebf5dfa24cdb98618ca5cc54cad4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:55.846246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=953aebf5dfa24cdb98618ca5cc54cad4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:55.850176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:55.915102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953aebf5dfa24cdb98618ca5cc54cad4', 'request_id': 'req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.925427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953aebf5dfa24cdb98618ca5cc54cad4', 'request_id': 'req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.935610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953aebf5dfa24cdb98618ca5cc54cad4', 'request_id': 'req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.956643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] The token cache is being invalidate because project 953aebf5dfa24cdb98618ca5cc54cad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:32:55.957114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.958083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3c92402-34f9-4283-a6f3-10f0a3fa8e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:32:55.960227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1174/2361] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:32:55 2022] DELETE /identity/v3/projects/953aebf5dfa24cdb98618ca5cc54cad4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:55.966584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932e4a0-6a1a-43ad-b9f5-05d6e3f89d45 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:55.967416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932e4a0-6a1a-43ad-b9f5-05d6e3f89d45 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:55.967558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932e4a0-6a1a-43ad-b9f5-05d6e3f89d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:56.027169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d932e4a0-6a1a-43ad-b9f5-05d6e3f89d45 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:56.029699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d932e4a0-6a1a-43ad-b9f5-05d6e3f89d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:56.262700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1188/2362] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:32:56.278321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6994c-c620-496a-96e1-e95a6ba55d52 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:56.279167 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6994c-c620-496a-96e1-e95a6ba55d52 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:56.279523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6994c-c620-496a-96e1-e95a6ba55d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:56.343852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f2c6994c-c620-496a-96e1-e95a6ba55d52 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:56.349280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f2c6994c-c620-496a-96e1-e95a6ba55d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:56.363536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1175/2363] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:56.372373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932e4a0-6a1a-43ad-b9f5-05d6e3f89d45 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:56.395461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932e4a0-6a1a-43ad-b9f5-05d6e3f89d45 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:56.402181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:56.404039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:56.404213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:56.404491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:56.405607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:32:56.406124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:56.408196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:56.425889 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1189/2364] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:32:56.432717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6994c-c620-496a-96e1-e95a6ba55d52 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:56.446268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6994c-c620-496a-96e1-e95a6ba55d52 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:56.453923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3db0bb-ae14-4b13-8b15-7dc3d0b81b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:56.454621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3db0bb-ae14-4b13-8b15-7dc3d0b81b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:56.454915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3db0bb-ae14-4b13-8b15-7dc3d0b81b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:56.455198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3db0bb-ae14-4b13-8b15-7dc3d0b81b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:56.455881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3db0bb-ae14-4b13-8b15-7dc3d0b81b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:56.458695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3db0bb-ae14-4b13-8b15-7dc3d0b81b2d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:56.512785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1176/2365] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:32:56 2022] POST /identity/v3/projects => generated 397 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:32:56.519063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:56.539402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32eac3-0631-4a0d-b059-07d687674e4f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:56.653881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b732e-55fe-4b21-9e0c-37466d379e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:56.655459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b732e-55fe-4b21-9e0c-37466d379e39 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:56.656056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b732e-55fe-4b21-9e0c-37466d379e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:56.657598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b732e-55fe-4b21-9e0c-37466d379e39 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:56.659112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b732e-55fe-4b21-9e0c-37466d379e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:56.663680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b732e-55fe-4b21-9e0c-37466d379e39 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:56.706741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1190/2366] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:32:56 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:32:56.712211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3db0bb-ae14-4b13-8b15-7dc3d0b81b2d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:56.733929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3db0bb-ae14-4b13-8b15-7dc3d0b81b2d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:56.749395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba55a9-8c78-416f-9172-e7e131a470ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:56.750263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba55a9-8c78-416f-9172-e7e131a470ec admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:56.750640 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba55a9-8c78-416f-9172-e7e131a470ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:56.751187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba55a9-8c78-416f-9172-e7e131a470ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:56.752174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba55a9-8c78-416f-9172-e7e131a470ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:56.754844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba55a9-8c78-416f-9172-e7e131a470ec admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:56.768362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1177/2367] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:56.773217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b732e-55fe-4b21-9e0c-37466d379e39 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:56.785350 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b732e-55fe-4b21-9e0c-37466d379e39 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:56.795043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1774bd-1464-46d5-be59-b9a2b868d771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:56.795912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1774bd-1464-46d5-be59-b9a2b868d771 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:56.796270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1774bd-1464-46d5-be59-b9a2b868d771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:56.796648 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1774bd-1464-46d5-be59-b9a2b868d771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:56.797440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1774bd-1464-46d5-be59-b9a2b868d771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:56.801104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1774bd-1464-46d5-be59-b9a2b868d771 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:56.815973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1191/2368] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:32:56.820938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba55a9-8c78-416f-9172-e7e131a470ec admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:56.842931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba55a9-8c78-416f-9172-e7e131a470ec admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:56.855987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:56.857233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:56.857745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:56.858249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] PATH_INFO: `/v3/projects/5516e52a3d4a450ba5d546d421e6a096/users/65672e04884d495ba9436d8e391f6946/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:56.900571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5516e52a3d4a450ba5d546d421e6a096, user_id=65672e04884d495ba9436d8e391f6946, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:56.907797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:56.948735 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1178/2369] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:32:56 2022] PUT /identity/v3/projects/5516e52a3d4a450ba5d546d421e6a096/users/65672e04884d495ba9436d8e391f6946/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:32:56.959311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd27954-131e-4aaa-9265-939c24666e1c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:56.959776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd27954-131e-4aaa-9265-939c24666e1c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:56.960252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd27954-131e-4aaa-9265-939c24666e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:57.044274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bbd27954-131e-4aaa-9265-939c24666e1c None None] MFA Rules not processed for user `65672e04884d495ba9436d8e391f6946`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:32:57.046488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bbd27954-131e-4aaa-9265-939c24666e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:57.135709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:32:57.189164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1192/2370] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:32:57.209508 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:32:57.211821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:32:57.355417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bcc993-0ba6-4bef-952b-e75519241691 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:32:57.361515 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee911810-59c2-41ff-b4e3-e46b93c9c843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:32:57.362141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee911810-59c2-41ff-b4e3-e46b93c9c843 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:32:57.362638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee911810-59c2-41ff-b4e3-e46b93c9c843 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:32:57.365050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee911810-59c2-41ff-b4e3-e46b93c9c843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:32:57.385185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee911810-59c2-41ff-b4e3-e46b93c9c843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:32:57.388136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee911810-59c2-41ff-b4e3-e46b93c9c843 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:32:57.407613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1179/2371] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:32:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:33:07.180963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=9 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:07.399057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:07.755560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd27954-131e-4aaa-9265-939c24666e1c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:33:07.841882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd27954-131e-4aaa-9265-939c24666e1c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:33:07.846776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:33:07.847450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:07.847726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:07.848010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] PATH_INFO: `/v3/users/65672e04884d495ba9436d8e391f6946` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:07.849253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=65672e04884d495ba9436d8e391f6946)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:33:07.851285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:33:07.933344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65672e04884d495ba9436d8e391f6946'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:07.958614 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65672e04884d495ba9436d8e391f6946'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:07.959862 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65672e04884d495ba9436d8e391f6946'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:07.973499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65672e04884d495ba9436d8e391f6946'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:07.990458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65672e04884d495ba9436d8e391f6946'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:08.005762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1193/2372] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:33:07 2022] DELETE /identity/v3/users/65672e04884d495ba9436d8e391f6946 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:33:08.013398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c390a-94f3-489b-8b7a-3c2fda78be73 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:08.013777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c390a-94f3-489b-8b7a-3c2fda78be73 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:08.014127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c390a-94f3-489b-8b7a-3c2fda78be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:08.046175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e95c390a-94f3-489b-8b7a-3c2fda78be73 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:33:08.047775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:33:08.049092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e95c390a-94f3-489b-8b7a-3c2fda78be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:08.050792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:08.186840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e6aea-ee31-4af1-825b-6decd9a08c8f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:33:08.196220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:33:08.196769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:08.197363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:08.197910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] PATH_INFO: `/v3/users/4eb3df7400404108b9a215fb69f0bf3f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:08.200296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb3df7400404108b9a215fb69f0bf3f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:33:08.203087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1180/2373] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:33:08.207134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:33:08.235264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c390a-94f3-489b-8b7a-3c2fda78be73 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:33:08.237693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e95c390a-94f3-489b-8b7a-3c2fda78be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:08.256165 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:08.296033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb3df7400404108b9a215fb69f0bf3f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:08.319522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb3df7400404108b9a215fb69f0bf3f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:08.320979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb3df7400404108b9a215fb69f0bf3f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:08.349427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c390a-94f3-489b-8b7a-3c2fda78be73 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:33:08.350671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb3df7400404108b9a215fb69f0bf3f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:08.354599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:33:08.356016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:08.356749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:08.357233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:08.360415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:08.363758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a92aeb4e-2af3-49e4-8d70-c74e221fab1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb3df7400404108b9a215fb69f0bf3f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:08.370998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1194/2374] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:33:08 2022] DELETE /identity/v3/users/4eb3df7400404108b9a215fb69f0bf3f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:33:08.375735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e8c46-09da-4adb-b461-53d8bb858237 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:08.376061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e8c46-09da-4adb-b461-53d8bb858237 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:08.376351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e8c46-09da-4adb-b461-53d8bb858237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:08.383693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:33:08.386755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:33:08.397962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ba9e8c46-09da-4adb-b461-53d8bb858237 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:33:08.399221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ba9e8c46-09da-4adb-b461-53d8bb858237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:08.406610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1181/2375] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:33:08.529977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1195/2376] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:33:08.554511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:33:08.557006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:08.582268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5a6a0-0385-4aa8-9796-a825a0a45798 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:33:08.697561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:33:08.698606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:08.699028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:08.699551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:08.719111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:33:08.720792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:33:08.741062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1182/2377] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:33:09.008424 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e8c46-09da-4adb-b461-53d8bb858237 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:33:09.012268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ba9e8c46-09da-4adb-b461-53d8bb858237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:09.040607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e8c46-09da-4adb-b461-53d8bb858237 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:33:09.054767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:33:09.055815 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:09.057328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:09.057328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] PATH_INFO: `/v3/projects/5516e52a3d4a450ba5d546d421e6a096` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:09.059998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5516e52a3d4a450ba5d546d421e6a096)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:33:09.062828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:33:09.210290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5516e52a3d4a450ba5d546d421e6a096', 'request_id': 'req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.222757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:33:09.228808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:09.230856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5516e52a3d4a450ba5d546d421e6a096', 'request_id': 'req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.257997 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5516e52a3d4a450ba5d546d421e6a096', 'request_id': 'req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.270028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f161c8-2bec-44fa-aeaf-bf719698d8a8 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:33:09.300195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] The token cache is being invalidate because project 5516e52a3d4a450ba5d546d421e6a096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:33:09.303033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.304793 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-856ba17c-1cc6-4edb-ac62-f4e0e1dc5319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.325487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1196/2378] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:33:09 2022] DELETE /identity/v3/projects/5516e52a3d4a450ba5d546d421e6a096 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:33:09.487143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:09.528207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:33:09.528984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:09.530337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:09.530337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] PATH_INFO: `/v3/projects/f5a68ad78ec547dcb0036549fbca1d49` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:09.531566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a68ad78ec547dcb0036549fbca1d49)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:33:09.533929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:33:09.640578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a68ad78ec547dcb0036549fbca1d49', 'request_id': 'req-11a3a68f-9935-42f4-85d3-80d6b88aaadd'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.654081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a68ad78ec547dcb0036549fbca1d49', 'request_id': 'req-11a3a68f-9935-42f4-85d3-80d6b88aaadd'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.676801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a68ad78ec547dcb0036549fbca1d49', 'request_id': 'req-11a3a68f-9935-42f4-85d3-80d6b88aaadd'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.696774 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] The token cache is being invalidate because project f5a68ad78ec547dcb0036549fbca1d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:33:09.698364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.699422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-11a3a68f-9935-42f4-85d3-80d6b88aaadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:33:09.703950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1183/2379] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:33:09 2022] DELETE /identity/v3/projects/f5a68ad78ec547dcb0036549fbca1d49 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:33:19.277806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=14,keystone:SELECT=74,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:19.491321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e2754-c3ee-4ba2-ac9b-2b848c06a5c8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:19.491656 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e2754-c3ee-4ba2-ac9b-2b848c06a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:19.492022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e2754-c3ee-4ba2-ac9b-2b848c06a5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:19.517439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-643e2754-c3ee-4ba2-ac9b-2b848c06a5c8 None None] MFA Rules not processed for user `9b293568e74c4c69baee5cb29013ab6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:33:19.518868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-643e2754-c3ee-4ba2-ac9b-2b848c06a5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:19.595583 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1197/2380] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:33:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:33:19.693336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:29.551873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3d9d-2f1c-4ea7-9271-c829f0e0d5e5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:29.552556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3d9d-2f1c-4ea7-9271-c829f0e0d5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:29.552768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3d9d-2f1c-4ea7-9271-c829f0e0d5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:29.583491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=25 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:29.590754 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-74ce3d9d-2f1c-4ea7-9271-c829f0e0d5e5 None None] MFA Rules not processed for user `98de1ccbabc64162a29cbce2cfe11600`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:33:29.593199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74ce3d9d-2f1c-4ea7-9271-c829f0e0d5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:29.664674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1184/2381] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:33:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:33:39.655565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=25 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:41.121182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc69f-a280-446e-806e-fad858217cc8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:41.121984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc69f-a280-446e-806e-fad858217cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:41.122393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc69f-a280-446e-806e-fad858217cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:41.145610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-79efc69f-a280-446e-806e-fad858217cc8 None None] MFA Rules not processed for user `ae71fd7ba4334d0d813cbc0c06e3686e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:33:41.146867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-79efc69f-a280-446e-806e-fad858217cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:41.211936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1198/2382] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:33:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:33:41.711085 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ed06-083e-4c8c-9e3e-449241e451c7 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:41.712481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ed06-083e-4c8c-9e3e-449241e451c7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:41.712481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ed06-083e-4c8c-9e3e-449241e451c7 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:41.716454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1185/2383] 10.222.0.58 () {54 vars in 961 bytes} [Mon Oct 3 13:33:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 03 13:33:41.724736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c476c4-ab4d-4528-bdb1-f1c45b9a7715 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:41.725106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c476c4-ab4d-4528-bdb1-f1c45b9a7715 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:41.725298 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c476c4-ab4d-4528-bdb1-f1c45b9a7715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:41.765234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-35c476c4-ab4d-4528-bdb1-f1c45b9a7715 None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:33:41.766854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-35c476c4-ab4d-4528-bdb1-f1c45b9a7715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:41.883031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1199/2384] 10.222.0.58 () {58 vars in 1058 bytes} [Mon Oct 3 13:33:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:33:41.893094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89ed06-083e-4c8c-9e3e-449241e451c7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:33:41.920949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89ed06-083e-4c8c-9e3e-449241e451c7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:33:41.937877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:33:41.937877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:33:41.937877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:33:41.937877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:33:41.953192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:33:41.982018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:33:41.983762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:33:42.006236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1186/2385] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:33:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:33:51.864033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=54 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:33:51.995911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:34:05.068749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9373fb-ec9e-4b13-9dc9-b153dee77e38 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:05.069435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9373fb-ec9e-4b13-9dc9-b153dee77e38 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:05.069435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9373fb-ec9e-4b13-9dc9-b153dee77e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:05.098123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bf9373fb-ec9e-4b13-9dc9-b153dee77e38 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:34:05.099391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bf9373fb-ec9e-4b13-9dc9-b153dee77e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:05.179572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1200/2386] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:34:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:34:05.235308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:05.236784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:05.329618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a33d6-9967-4632-9cc0-acbf6d71ea03 service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:05.335960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c568e1-bb9c-4a99-ba09-39894258060f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:05.336906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c568e1-bb9c-4a99-ba09-39894258060f service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:05.337290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c568e1-bb9c-4a99-ba09-39894258060f service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:05.338339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c568e1-bb9c-4a99-ba09-39894258060f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:05.350089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c568e1-bb9c-4a99-ba09-39894258060f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:05.352092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c568e1-bb9c-4a99-ba09-39894258060f service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:05.370906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1187/2387] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:34:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:34:15.171548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=25 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:34:15.360008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=32 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:34:31.093580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14992f6a-77a9-4cc4-b5ed-b4f4c43a236e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:31.094268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14992f6a-77a9-4cc4-b5ed-b4f4c43a236e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:31.094268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14992f6a-77a9-4cc4-b5ed-b4f4c43a236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:31.119284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-14992f6a-77a9-4cc4-b5ed-b4f4c43a236e None None] MFA Rules not processed for user `75e5a47c5b314a0486d3a32ffbead793`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:34:31.122096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-14992f6a-77a9-4cc4-b5ed-b4f4c43a236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:31.212640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1201/2388] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:34:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:34:31.580882 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7e9e6-a07a-44d0-be1a-b716cb29c6bb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:31.581500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7e9e6-a07a-44d0-be1a-b716cb29c6bb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:31.581500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7e9e6-a07a-44d0-be1a-b716cb29c6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:31.603606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a6b7e9e6-a07a-44d0-be1a-b716cb29c6bb None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:34:31.604814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a6b7e9e6-a07a-44d0-be1a-b716cb29c6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:31.612286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1188/2389] 10.222.0.58 () {58 vars in 967 bytes} [Mon Oct 3 13:34:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:34:31.626070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14992f6a-77a9-4cc4-b5ed-b4f4c43a236e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:31.639446 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14992f6a-77a9-4cc4-b5ed-b4f4c43a236e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:31.644691 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e575d2-cc6b-4fda-a4d6-208542e4d6da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:31.645821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e575d2-cc6b-4fda-a4d6-208542e4d6da service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:31.645821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e575d2-cc6b-4fda-a4d6-208542e4d6da service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:31.646056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e575d2-cc6b-4fda-a4d6-208542e4d6da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:31.655725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e575d2-cc6b-4fda-a4d6-208542e4d6da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:31.657328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e575d2-cc6b-4fda-a4d6-208542e4d6da service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:31.667355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1202/2390] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:34:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:34:34.179035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ff64-1bf3-4551-b8f0-d2a653261b9a None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:34.179580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ff64-1bf3-4551-b8f0-d2a653261b9a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:34.179580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ff64-1bf3-4551-b8f0-d2a653261b9a None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:34.180936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1189/2391] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:34:34.194813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1e124-1a45-4e39-a5e2-f4ff647f73f3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:34.195375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1e124-1a45-4e39-a5e2-f4ff647f73f3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:34.195542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1e124-1a45-4e39-a5e2-f4ff647f73f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:34.218461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f7d1e124-1a45-4e39-a5e2-f4ff647f73f3 None None] MFA Rules not processed for user `7bca29b8758b408da2bf05face26b551`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:34:34.219820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f7d1e124-1a45-4e39-a5e2-f4ff647f73f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:34.231589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1203/2392] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:34:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:34:34.269565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7ff64-1bf3-4551-b8f0-d2a653261b9a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:34.275720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c1a7ff64-1bf3-4551-b8f0-d2a653261b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:34.303159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7ff64-1bf3-4551-b8f0-d2a653261b9a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:34.312470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03940a8-0d75-4743-86fb-5798088567d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:34.313221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03940a8-0d75-4743-86fb-5798088567d5 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:34.313675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03940a8-0d75-4743-86fb-5798088567d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:34.313934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03940a8-0d75-4743-86fb-5798088567d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:34.319917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1e124-1a45-4e39-a5e2-f4ff647f73f3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:34.321635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f7d1e124-1a45-4e39-a5e2-f4ff647f73f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:34.324569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03940a8-0d75-4743-86fb-5798088567d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:34.326746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03940a8-0d75-4743-86fb-5798088567d5 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:34.347450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1190/2393] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:34:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:34:34.444561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1e124-1a45-4e39-a5e2-f4ff647f73f3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:34.450115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:34.450882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:34.451245 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:34.451598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] PATH_INFO: `/v3/users/9b293568e74c4c69baee5cb29013ab6d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:34.453172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b293568e74c4c69baee5cb29013ab6d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:34.455982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:34.520329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b293568e74c4c69baee5cb29013ab6d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:34.537782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b293568e74c4c69baee5cb29013ab6d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:34.538838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b293568e74c4c69baee5cb29013ab6d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:34.550442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b293568e74c4c69baee5cb29013ab6d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:34.562857 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c57ceb71-8438-4bf5-a39e-bb5910cf061e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b293568e74c4c69baee5cb29013ab6d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:34.576266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1204/2394] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:34:34 2022] DELETE /identity/v3/users/9b293568e74c4c69baee5cb29013ab6d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:34:34.886671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03940a8-0d75-4743-86fb-5798088567d5 service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:34.907925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03940a8-0d75-4743-86fb-5798088567d5 service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:34.988778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:34.989344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:34.989652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:34.989931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] PATH_INFO: `/v3/projects/afa1958afb874a0cba4c57df10f2ce08` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:34.991184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa1958afb874a0cba4c57df10f2ce08)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:34.993157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:35.055285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa1958afb874a0cba4c57df10f2ce08', 'request_id': 'req-9e283357-108f-4247-b14b-cc76709c44f5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:35.063689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa1958afb874a0cba4c57df10f2ce08', 'request_id': 'req-9e283357-108f-4247-b14b-cc76709c44f5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:35.074492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa1958afb874a0cba4c57df10f2ce08', 'request_id': 'req-9e283357-108f-4247-b14b-cc76709c44f5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:35.093486 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] The token cache is being invalidate because project afa1958afb874a0cba4c57df10f2ce08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:34:35.093924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:35.094783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9e283357-108f-4247-b14b-cc76709c44f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:34:35.096803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1191/2395] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:34:34 2022] DELETE /identity/v3/projects/afa1958afb874a0cba4c57df10f2ce08 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:34:35.108900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbac80-d051-4797-bc7b-3ed0b14c0d27 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.109208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbac80-d051-4797-bc7b-3ed0b14c0d27 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.109519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbac80-d051-4797-bc7b-3ed0b14c0d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.118297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:34:35.132964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b6bbac80-d051-4797-bc7b-3ed0b14c0d27 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:34:35.141171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b6bbac80-d051-4797-bc7b-3ed0b14c0d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:35.248072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1205/2396] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:34:35.315786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb714-2188-4ade-b1d1-f54693effa4a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.317005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb714-2188-4ade-b1d1-f54693effa4a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.317469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb714-2188-4ade-b1d1-f54693effa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.348166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7b9cb714-2188-4ade-b1d1-f54693effa4a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:34:35.349785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7b9cb714-2188-4ade-b1d1-f54693effa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:35.360459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1192/2397] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:34:35.364471 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbac80-d051-4797-bc7b-3ed0b14c0d27 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:35.377753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbac80-d051-4797-bc7b-3ed0b14c0d27 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:35.388333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:35.390012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.390299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.390705 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.391945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:34:35.392654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:35.402874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:35.426928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1206/2398] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:34:35.432773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cb714-2188-4ade-b1d1-f54693effa4a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:35.446031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cb714-2188-4ade-b1d1-f54693effa4a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:35.452692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acaea1-a464-47ba-a99a-b5766be80d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:35.453672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acaea1-a464-47ba-a99a-b5766be80d68 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.454168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acaea1-a464-47ba-a99a-b5766be80d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.454660 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acaea1-a464-47ba-a99a-b5766be80d68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.455813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acaea1-a464-47ba-a99a-b5766be80d68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:35.459246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acaea1-a464-47ba-a99a-b5766be80d68 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:35.493951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1193/2399] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:34:35 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:34:35.498605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:35.513927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502f65a-33ed-4dad-9269-54bfdc0c3068 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:35.603139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ae8b-025d-4cfb-999b-7ead698dfcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:35.603376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ae8b-025d-4cfb-999b-7ead698dfcff admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.604015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ae8b-025d-4cfb-999b-7ead698dfcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.604015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9ae8b-025d-4cfb-999b-7ead698dfcff admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.604466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9ae8b-025d-4cfb-999b-7ead698dfcff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:35.606971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9ae8b-025d-4cfb-999b-7ead698dfcff admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:35.632609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1207/2400] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:34:35 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:34:35.637713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acaea1-a464-47ba-a99a-b5766be80d68 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:35.653173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acaea1-a464-47ba-a99a-b5766be80d68 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:35.659579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082feed5-e7b3-4d59-bc5b-63e72bee32ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:35.660393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082feed5-e7b3-4d59-bc5b-63e72bee32ac admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.660840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082feed5-e7b3-4d59-bc5b-63e72bee32ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.661358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082feed5-e7b3-4d59-bc5b-63e72bee32ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.662057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082feed5-e7b3-4d59-bc5b-63e72bee32ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:35.663767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082feed5-e7b3-4d59-bc5b-63e72bee32ac admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:35.679064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1194/2401] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:34:35.684367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ae8b-025d-4cfb-999b-7ead698dfcff admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:35.696824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9ae8b-025d-4cfb-999b-7ead698dfcff admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:35.702547 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057741c0-3410-45ee-af1d-e1921f9d4561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:35.703172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057741c0-3410-45ee-af1d-e1921f9d4561 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.703458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057741c0-3410-45ee-af1d-e1921f9d4561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.703736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057741c0-3410-45ee-af1d-e1921f9d4561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.704400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057741c0-3410-45ee-af1d-e1921f9d4561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:35.706041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057741c0-3410-45ee-af1d-e1921f9d4561 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:35.717341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1208/2402] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:34:35.721766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082feed5-e7b3-4d59-bc5b-63e72bee32ac admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:35.732134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082feed5-e7b3-4d59-bc5b-63e72bee32ac admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:35.737052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:35.737751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.738029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.738301 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] PATH_INFO: `/v3/projects/ecc7724583e6494198bf1a0bd8181d2c/users/a1f55d4b68c9499fbaf11bd2eadd6f53/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.755165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc7724583e6494198bf1a0bd8181d2c, user_id=a1f55d4b68c9499fbaf11bd2eadd6f53, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:35.758354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:35.772954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1195/2403] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:34:35 2022] PUT /identity/v3/projects/ecc7724583e6494198bf1a0bd8181d2c/users/a1f55d4b68c9499fbaf11bd2eadd6f53/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:34:35.787341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d1ea-92e7-4539-8a64-ac0fa8f1129a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.787653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d1ea-92e7-4539-8a64-ac0fa8f1129a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.787954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02d1ea-92e7-4539-8a64-ac0fa8f1129a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:35.823345 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8b02d1ea-92e7-4539-8a64-ac0fa8f1129a None None] MFA Rules not processed for user `a1f55d4b68c9499fbaf11bd2eadd6f53`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:34:35.825437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8b02d1ea-92e7-4539-8a64-ac0fa8f1129a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:35.895551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:34:35.901051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1209/2404] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:34:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:34:35.914044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:34:35.916125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:34:35.930297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:34:35.992347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f595b-4bc6-4e55-a75b-f54a3f97eac8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:34:35.996312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e974c28-894e-4fb9-863d-9c209219b495 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '464190e0ccaf4f82942f518df1bf6c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:34:35.996930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e974c28-894e-4fb9-863d-9c209219b495 service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:34:35.997214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e974c28-894e-4fb9-863d-9c209219b495 service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:34:35.997521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e974c28-894e-4fb9-863d-9c209219b495 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:34:36.007792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e974c28-894e-4fb9-863d-9c209219b495 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:34:36.009177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e974c28-894e-4fb9-863d-9c209219b495 service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:34:36.021953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1196/2405] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:34:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:34:46.016213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=23 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:35:29.386730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02d1ea-92e7-4539-8a64-ac0fa8f1129a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:29.388277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8b02d1ea-92e7-4539-8a64-ac0fa8f1129a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:29.453785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02d1ea-92e7-4539-8a64-ac0fa8f1129a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:29.458381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e21b2a-f8cd-4dfb-9456-78c15e8b1459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:29.458995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e21b2a-f8cd-4dfb-9456-78c15e8b1459 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:29.459330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e21b2a-f8cd-4dfb-9456-78c15e8b1459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:29.459674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e21b2a-f8cd-4dfb-9456-78c15e8b1459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:29.461410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c4e21b2a-f8cd-4dfb-9456-78c15e8b1459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:29.552176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e21b2a-f8cd-4dfb-9456-78c15e8b1459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:29.553645 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e21b2a-f8cd-4dfb-9456-78c15e8b1459 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:29.566508 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1210/2406] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:35:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:35:29.609422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e974c28-894e-4fb9-863d-9c209219b495 service glance] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:29.690747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e974c28-894e-4fb9-863d-9c209219b495 service glance] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:29.696533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:29.697504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:29.697844 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:29.698337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] PATH_INFO: `/v3/users/a1f55d4b68c9499fbaf11bd2eadd6f53` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:29.700080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f55d4b68c9499fbaf11bd2eadd6f53)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:29.702001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:29.761090 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f55d4b68c9499fbaf11bd2eadd6f53'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:29.761777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f55d4b68c9499fbaf11bd2eadd6f53'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:29.762622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f55d4b68c9499fbaf11bd2eadd6f53'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:29.771631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f55d4b68c9499fbaf11bd2eadd6f53'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:29.784764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f55d4b68c9499fbaf11bd2eadd6f53'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:29.798829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1197/2407] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:35:29 2022] DELETE /identity/v3/users/a1f55d4b68c9499fbaf11bd2eadd6f53 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:35:29.803562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280f8b4-f5d8-40b5-8bc8-d40b40953f8f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:29.803856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280f8b4-f5d8-40b5-8bc8-d40b40953f8f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:29.804140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280f8b4-f5d8-40b5-8bc8-d40b40953f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:29.827434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6280f8b4-f5d8-40b5-8bc8-d40b40953f8f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:35:29.828791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6280f8b4-f5d8-40b5-8bc8-d40b40953f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:29.902102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1211/2408] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:35:29.912123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:29.913600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:29.975515 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f82dd-9b7b-4e7c-bd95-093e7dd7a9d4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:29.979550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:29.980670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:29.980670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:29.980881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:29.990504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:29.991941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:30.002332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1198/2409] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:35:30.322036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280f8b4-f5d8-40b5-8bc8-d40b40953f8f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:30.323885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6280f8b4-f5d8-40b5-8bc8-d40b40953f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:30.341905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280f8b4-f5d8-40b5-8bc8-d40b40953f8f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:30.348618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:30.349281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:30.349595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:30.349922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] PATH_INFO: `/v3/projects/ecc7724583e6494198bf1a0bd8181d2c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:30.351109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc7724583e6494198bf1a0bd8181d2c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:30.353671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:30.402180 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:35:30.411134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc7724583e6494198bf1a0bd8181d2c', 'request_id': 'req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:30.418604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc7724583e6494198bf1a0bd8181d2c', 'request_id': 'req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:30.428450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc7724583e6494198bf1a0bd8181d2c', 'request_id': 'req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:30.441884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] The token cache is being invalidate because project ecc7724583e6494198bf1a0bd8181d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:35:30.442390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:30.443065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:30.444550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1212/2410] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:35:30 2022] DELETE /identity/v3/projects/ecc7724583e6494198bf1a0bd8181d2c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:35:31.054590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:31.056179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:31.067609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72450f9a-adaf-471b-a8ea-4e780affc807 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:31.086385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:35:31.120422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:31.121225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:31.121546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:31.121823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:31.123567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:31.196181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:31.198257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:31.209286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1199/2411] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:35:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:35:40.438549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:35:41.204476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=44 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:35:49.998051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:50.000013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:50.017201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd4a7b-9570-49ab-bbd6-38a1e4cc1eef admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:50.023474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:50.024089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:50.024406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:50.026498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:50.028445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:50.090597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:50.092255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:50.101675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1213/2412] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:35:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:35:51.139674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:51.141101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:51.202905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67f983-96e2-4766-b83b-b6021bdfba3b service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:51.206701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:51.207320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:51.207605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:51.207880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:51.209659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:51.268889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:51.270346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:51.280032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1200/2413] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:35:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:35:56.350650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:56.352266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:35:56.425534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b665a-ab98-4e32-a4bc-698e60586b59 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:56.430214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:56.430873 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:56.431159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:56.431432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] PATH_INFO: `/v3/users/75e5a47c5b314a0486d3a32ffbead793` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:56.432590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e5a47c5b314a0486d3a32ffbead793)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:56.434952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:56.493407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e5a47c5b314a0486d3a32ffbead793'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.507447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e5a47c5b314a0486d3a32ffbead793'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.508199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e5a47c5b314a0486d3a32ffbead793'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.516662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e5a47c5b314a0486d3a32ffbead793'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.523974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-53150648-9d05-4a78-b9f9-47ae4c22d1ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e5a47c5b314a0486d3a32ffbead793'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.530824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1214/2414] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:35:56 2022] DELETE /identity/v3/users/75e5a47c5b314a0486d3a32ffbead793 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:35:56.724145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:35:56.743165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69427643-afce-4a3a-a5ba-c1589bdd264b service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:35:56.844514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:35:56.845662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:35:56.846179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:35:56.846695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] PATH_INFO: `/v3/projects/33af9400d270489da18c21dee3f9188f` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:35:56.848688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33af9400d270489da18c21dee3f9188f)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:35:56.852614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:35:56.911597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33af9400d270489da18c21dee3f9188f', 'request_id': 'req-614af925-9964-4805-8748-7e8f5b9d4752'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.919728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33af9400d270489da18c21dee3f9188f', 'request_id': 'req-614af925-9964-4805-8748-7e8f5b9d4752'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.921904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:35:56.926969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33af9400d270489da18c21dee3f9188f', 'request_id': 'req-614af925-9964-4805-8748-7e8f5b9d4752'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.941733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] The token cache is being invalidate because project 33af9400d270489da18c21dee3f9188f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:35:56.942031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.942838 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:35:56.944365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1201/2415] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:35:56 2022] DELETE /identity/v3/projects/33af9400d270489da18c21dee3f9188f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:36:01.789688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e83d8-ca1e-497c-ae55-8364a351d9e5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:36:01.790229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e83d8-ca1e-497c-ae55-8364a351d9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:36:01.790229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e83d8-ca1e-497c-ae55-8364a351d9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:36:01.810709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a75e83d8-ca1e-497c-ae55-8364a351d9e5 None None] MFA Rules not processed for user `98de1ccbabc64162a29cbce2cfe11600`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:36:01.811988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a75e83d8-ca1e-497c-ae55-8364a351d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:01.826540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:36:01.878188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1215/2416] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:36:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:36:02.224033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:36:02.225715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:02.290462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614af925-9964-4805-8748-7e8f5b9d4752 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:36:02.294353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:36:02.294982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:36:02.295275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:36:02.295551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:36:02.297337 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:02.307788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:36:02.309251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:36:02.317913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1202/2417] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:36:02.654872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e83d8-ca1e-497c-ae55-8364a351d9e5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:36:02.656297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a75e83d8-ca1e-497c-ae55-8364a351d9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:02.736387 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e83d8-ca1e-497c-ae55-8364a351d9e5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:36:02.741572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd9177-355e-4486-ac01-a3166218cc8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:36:02.742186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd9177-355e-4486-ac01-a3166218cc8f service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:36:02.742473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd9177-355e-4486-ac01-a3166218cc8f service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:36:02.742759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd9177-355e-4486-ac01-a3166218cc8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:36:02.744530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-49dd9177-355e-4486-ac01-a3166218cc8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:02.757053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd9177-355e-4486-ac01-a3166218cc8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:36:02.758672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd9177-355e-4486-ac01-a3166218cc8f service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:36:02.769647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1216/2418] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:36:09.467903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:36:09.477020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1136838-178e-46df-a335-2bc078f90c5c service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:36:09.480989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:36:09.481643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:36:09.481931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:36:09.482210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:36:09.484345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:09.498885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:36:09.500932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:36:09.512281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1203/2419] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:36:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:36:12.765097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=49 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:36:18.909378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacec3a-cd2d-4034-bc3f-d93ded1ca975 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:36:18.910027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacec3a-cd2d-4034-bc3f-d93ded1ca975 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:36:18.910027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacec3a-cd2d-4034-bc3f-d93ded1ca975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:36:18.929319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ddacec3a-cd2d-4034-bc3f-d93ded1ca975 None None] MFA Rules not processed for user `ae71fd7ba4334d0d813cbc0c06e3686e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:36:18.930572 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ddacec3a-cd2d-4034-bc3f-d93ded1ca975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:18.986008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1217/2420] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:36:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:36:19.506478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=36,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:36:23.910041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:36:23.912431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:23.992841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58dcbc7-4398-406f-bdd1-461aac46bb3b service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:36:23.998871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:36:23.999517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:36:23.999799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:36:24.000073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:36:24.002091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:36:24.016061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:36:24.017696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:36:24.031236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1204/2421] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:36:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:36:28.979097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=25 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:36:34.024673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=34 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:37:31.290056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b38e8e-e6e6-47ea-9d1f-574c4a4e8ed3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:31.290056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b38e8e-e6e6-47ea-9d1f-574c4a4e8ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:31.290056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b38e8e-e6e6-47ea-9d1f-574c4a4e8ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:31.307776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-84b38e8e-e6e6-47ea-9d1f-574c4a4e8ed3 None None] MFA Rules not processed for user `98de1ccbabc64162a29cbce2cfe11600`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:37:31.308987 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-84b38e8e-e6e6-47ea-9d1f-574c4a4e8ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:31.316477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1218/2422] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:37:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:37:39.308936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:39.310459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:39.322227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4839a-b90f-4448-9620-1d275e7ec518 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:39.325862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:39.326462 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:39.326746 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:39.327014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:39.328711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:39.402862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:39.404384 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:39.415185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1205/2423] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:37:41.282281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b38e8e-e6e6-47ea-9d1f-574c4a4e8ed3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:41.285436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-84b38e8e-e6e6-47ea-9d1f-574c4a4e8ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:41.432104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b38e8e-e6e6-47ea-9d1f-574c4a4e8ed3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:41.436146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:41.436795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:41.437081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:41.437363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:41.439366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:41.590888 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:41.592654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:41.618443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1219/2424] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:37:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:37:43.035927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:43.039254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:43.061164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c761-ad76-4155-97a8-a088ac4b8d2c service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:43.067998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:43.068584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.068915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.069136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] PATH_INFO: `/v3/users/ae71fd7ba4334d0d813cbc0c06e3686e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.070412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae71fd7ba4334d0d813cbc0c06e3686e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:43.072335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:43.122521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae71fd7ba4334d0d813cbc0c06e3686e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.123211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae71fd7ba4334d0d813cbc0c06e3686e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.123838 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae71fd7ba4334d0d813cbc0c06e3686e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.132119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae71fd7ba4334d0d813cbc0c06e3686e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.145035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-5227b54c-fed4-4f05-b5a9-6f4e4e15da3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae71fd7ba4334d0d813cbc0c06e3686e'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.160172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1206/2425] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:37:43 2022] DELETE /identity/v3/users/ae71fd7ba4334d0d813cbc0c06e3686e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:37:43.353565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:43.367306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd0438-e71b-41c1-a2a5-0054823271d8 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:43.439129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:43.440312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.440592 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.440867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] PATH_INFO: `/v3/projects/557321bf33ca42df8d5d53e5157e6803` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.442083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=557321bf33ca42df8d5d53e5157e6803)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:43.444063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:43.502854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557321bf33ca42df8d5d53e5157e6803', 'request_id': 'req-8a219e42-068c-4d59-8b62-ad8699f99465'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.504255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:37:43.510739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557321bf33ca42df8d5d53e5157e6803', 'request_id': 'req-8a219e42-068c-4d59-8b62-ad8699f99465'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.521259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557321bf33ca42df8d5d53e5157e6803', 'request_id': 'req-8a219e42-068c-4d59-8b62-ad8699f99465'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.535981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] The token cache is being invalidate because project 557321bf33ca42df8d5d53e5157e6803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:37:43.536279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.536973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8a219e42-068c-4d59-8b62-ad8699f99465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:37:43.538588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1220/2426] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:37:43 2022] DELETE /identity/v3/projects/557321bf33ca42df8d5d53e5157e6803 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:37:43.544320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce867dd2-2e2b-46d5-89e2-24868a43805a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.545869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce867dd2-2e2b-46d5-89e2-24868a43805a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.546246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce867dd2-2e2b-46d5-89e2-24868a43805a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.570309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ce867dd2-2e2b-46d5-89e2-24868a43805a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:37:43.571575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ce867dd2-2e2b-46d5-89e2-24868a43805a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:43.642348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1207/2427] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:37:43.650988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786baa70-46d7-4a26-ae93-b5a47d385a5f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.651339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786baa70-46d7-4a26-ae93-b5a47d385a5f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.651865 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786baa70-46d7-4a26-ae93-b5a47d385a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.678979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-786baa70-46d7-4a26-ae93-b5a47d385a5f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:37:43.680392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-786baa70-46d7-4a26-ae93-b5a47d385a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:43.689013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1221/2428] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:37:43.694368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce867dd2-2e2b-46d5-89e2-24868a43805a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:43.706899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce867dd2-2e2b-46d5-89e2-24868a43805a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:43.712716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:43.713403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.713724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.713988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.715001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:37:43.715001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:43.716730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:43.719736 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:37:43.736542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1208/2429] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:37:43.737861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786baa70-46d7-4a26-ae93-b5a47d385a5f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:43.747623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786baa70-46d7-4a26-ae93-b5a47d385a5f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:43.754182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d19f6d-cbcc-4303-83f1-45d110c0f93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:43.754825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19f6d-cbcc-4303-83f1-45d110c0f93e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.755262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19f6d-cbcc-4303-83f1-45d110c0f93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.755538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d19f6d-cbcc-4303-83f1-45d110c0f93e admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.756230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d19f6d-cbcc-4303-83f1-45d110c0f93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:43.758215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d19f6d-cbcc-4303-83f1-45d110c0f93e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:43.793091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1222/2430] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:37:43 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:37:43.794618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:43.806970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85909c6-7e08-4415-a839-5e6b8bd012d0 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:43.869897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a138fa-ca4a-4b92-895d-124a5ca6044c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:43.870524 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a138fa-ca4a-4b92-895d-124a5ca6044c admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.870814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a138fa-ca4a-4b92-895d-124a5ca6044c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.871423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a138fa-ca4a-4b92-895d-124a5ca6044c admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.871758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a138fa-ca4a-4b92-895d-124a5ca6044c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:43.873973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a138fa-ca4a-4b92-895d-124a5ca6044c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:43.898191 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1209/2431] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:37:43 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:37:43.902442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d19f6d-cbcc-4303-83f1-45d110c0f93e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:43.913431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d19f6d-cbcc-4303-83f1-45d110c0f93e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:43.918065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae79d4-442f-4486-bfc2-aec08b471b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:43.918675 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae79d4-442f-4486-bfc2-aec08b471b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.918951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae79d4-442f-4486-bfc2-aec08b471b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.919225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae79d4-442f-4486-bfc2-aec08b471b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.919874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae79d4-442f-4486-bfc2-aec08b471b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:43.921497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae79d4-442f-4486-bfc2-aec08b471b29 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:43.935942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1223/2432] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:37:43.937233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a138fa-ca4a-4b92-895d-124a5ca6044c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:43.949028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a138fa-ca4a-4b92-895d-124a5ca6044c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:43.955208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a618364-1d17-4adc-ba82-39aaa2bbd76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:43.956082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a618364-1d17-4adc-ba82-39aaa2bbd76b admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:43.956526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a618364-1d17-4adc-ba82-39aaa2bbd76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:43.956908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a618364-1d17-4adc-ba82-39aaa2bbd76b admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:43.958061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a618364-1d17-4adc-ba82-39aaa2bbd76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:43.960473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a618364-1d17-4adc-ba82-39aaa2bbd76b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:43.974276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1210/2433] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:37:43.978474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae79d4-442f-4486-bfc2-aec08b471b29 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:43.992378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae79d4-442f-4486-bfc2-aec08b471b29 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:44.007378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:44.008527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:44.008929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:44.010099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] PATH_INFO: `/v3/projects/e019f687769f408d9c3faab0c8693a9d/users/21fde7fa2a16426890bdcba2241778ed/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:44.039299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e019f687769f408d9c3faab0c8693a9d, user_id=21fde7fa2a16426890bdcba2241778ed, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:44.042091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:44.059947 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1224/2434] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:37:43 2022] PUT /identity/v3/projects/e019f687769f408d9c3faab0c8693a9d/users/21fde7fa2a16426890bdcba2241778ed/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:37:44.065304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ead0a6-6215-4b9c-a037-499ac5b16f7e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:44.065949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ead0a6-6215-4b9c-a037-499ac5b16f7e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:44.065949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ead0a6-6215-4b9c-a037-499ac5b16f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:44.089683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d8ead0a6-6215-4b9c-a037-499ac5b16f7e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:37:44.091021 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d8ead0a6-6215-4b9c-a037-499ac5b16f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:44.167009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1211/2435] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:37:44.178224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:44.179242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:44.249223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda8657-34b3-4fbf-8197-89ce0a4f26e8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:44.254467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52383e2-0a54-4db4-a0d5-ffba7cfb103d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:44.255101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52383e2-0a54-4db4-a0d5-ffba7cfb103d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:44.255389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52383e2-0a54-4db4-a0d5-ffba7cfb103d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:44.255671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52383e2-0a54-4db4-a0d5-ffba7cfb103d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:44.266537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52383e2-0a54-4db4-a0d5-ffba7cfb103d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:44.268244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52383e2-0a54-4db4-a0d5-ffba7cfb103d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:44.280301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1225/2436] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:37:46.004556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ead0a6-6215-4b9c-a037-499ac5b16f7e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:46.006117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d8ead0a6-6215-4b9c-a037-499ac5b16f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:46.153039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ead0a6-6215-4b9c-a037-499ac5b16f7e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:46.162154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:46.162957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:46.163315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:46.163791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:46.165928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:46.183288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:46.185483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:46.194945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:37:46.216385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1212/2437] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:37:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:37:49.522824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b8d8f-c019-4bda-89f2-ae9f957fa89b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:49.523335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b8d8f-c019-4bda-89f2-ae9f957fa89b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:49.523335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b8d8f-c019-4bda-89f2-ae9f957fa89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:49.599604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3e0b8d8f-c019-4bda-89f2-ae9f957fa89b None None] MFA Rules not processed for user `21fde7fa2a16426890bdcba2241778ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:37:49.601099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3e0b8d8f-c019-4bda-89f2-ae9f957fa89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:49.741554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1226/2438] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:37:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:37:49.757545 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:37:49.770289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940afb8d-d1b7-4b8f-812c-f3efb21df587 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:37:49.782102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d775bee-4324-4dcb-948c-67434c7d198c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:37:49.782629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d775bee-4324-4dcb-948c-67434c7d198c service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:49.782941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d775bee-4324-4dcb-948c-67434c7d198c service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:49.783784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d775bee-4324-4dcb-948c-67434c7d198c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:49.812202 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d775bee-4324-4dcb-948c-67434c7d198c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:37:49.813867 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d775bee-4324-4dcb-948c-67434c7d198c service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:37:49.827306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1213/2439] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:37:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:37:50.581607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9167f-3d53-454a-93b9-573ed46f1243 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:50.586868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9167f-3d53-454a-93b9-573ed46f1243 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:50.587882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9167f-3d53-454a-93b9-573ed46f1243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:50.604410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:37:50.613569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1cc9167f-3d53-454a-93b9-573ed46f1243 None None] MFA Rules not processed for user `4471a60040ca42649f2718fd46659e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:37:50.613569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1cc9167f-3d53-454a-93b9-573ed46f1243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:50.694041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1227/2440] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:37:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:37:52.794838 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45abf-eb5f-47f0-9976-6da002e8a42e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:37:52.795666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45abf-eb5f-47f0-9976-6da002e8a42e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:37:52.795666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb45abf-eb5f-47f0-9976-6da002e8a42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:37:52.830169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0bb45abf-eb5f-47f0-9976-6da002e8a42e None None] MFA Rules not processed for user `21fde7fa2a16426890bdcba2241778ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:37:52.831770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0bb45abf-eb5f-47f0-9976-6da002e8a42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:37:52.845376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1214/2441] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:37:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:38:00.686540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=22 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:38:02.823847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=10 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:38:08.084491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9167f-3d53-454a-93b9-573ed46f1243 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:38:08.098352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9167f-3d53-454a-93b9-573ed46f1243 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:38:08.104943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:38:08.105729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:38:08.106098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:38:08.106451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:38:08.108296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:38:08.123463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:38:08.124342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:38:08.137048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1228/2442] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:38:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:38:18.131570 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:38:42.841544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb45abf-eb5f-47f0-9976-6da002e8a42e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:38:42.842960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0bb45abf-eb5f-47f0-9976-6da002e8a42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:38:42.988135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb45abf-eb5f-47f0-9976-6da002e8a42e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:38:42.993355 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:38:42.994002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:38:42.994288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:38:42.994566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:38:42.996358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:38:43.009332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:38:43.010993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:38:43.021621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1215/2443] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:38:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:38:53.016898 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=63 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:38:56.569978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:38:56.617057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34779e55-8944-49f8-9b83-bdcb16b123d4 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:38:56.621120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:38:56.621783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:38:56.622074 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:38:56.622355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:38:56.624193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:38:56.664370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:38:56.665866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:38:56.686635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1229/2444] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:00.203891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:00.208154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:00.223579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7864a1-6c13-4b4d-9ea7-a316ed26e3dc service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:00.228271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:00.228914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:00.229198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:00.229501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] PATH_INFO: `/v3/users/98de1ccbabc64162a29cbce2cfe11600` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:00.230629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98de1ccbabc64162a29cbce2cfe11600)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:00.232601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:00.281712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98de1ccbabc64162a29cbce2cfe11600'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.282372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98de1ccbabc64162a29cbce2cfe11600'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.283636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98de1ccbabc64162a29cbce2cfe11600'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.291404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98de1ccbabc64162a29cbce2cfe11600'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.303840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b3bead4a-bc3f-44d1-9880-5e39399b5ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98de1ccbabc64162a29cbce2cfe11600'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.319817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1216/2445] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:00 2022] DELETE /identity/v3/users/98de1ccbabc64162a29cbce2cfe11600 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:00.518326 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:00.528440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6373c8-447a-4b5f-a885-ee588fdeebba service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:00.589022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:00.589628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:00.589909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:00.590206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] PATH_INFO: `/v3/projects/d8e6dd58ff43419fa5b945bd08ef2931` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:00.591501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e6dd58ff43419fa5b945bd08ef2931)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:00.593517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:00.648925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e6dd58ff43419fa5b945bd08ef2931', 'request_id': 'req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.654472 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e6dd58ff43419fa5b945bd08ef2931', 'request_id': 'req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.661940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e6dd58ff43419fa5b945bd08ef2931', 'request_id': 'req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.673553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] The token cache is being invalidate because project d8e6dd58ff43419fa5b945bd08ef2931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:00.673846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.674496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1b4d5c6f-571e-4753-b5b5-d6a1b2fdfaec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:00.675981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1230/2446] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:00 2022] DELETE /identity/v3/projects/d8e6dd58ff43419fa5b945bd08ef2931 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:00.681025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cb66-922a-4994-8bd6-68c559c443fa None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:00.681321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cb66-922a-4994-8bd6-68c559c443fa None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:00.681650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6cb66-922a-4994-8bd6-68c559c443fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:00.741704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c4d6cb66-922a-4994-8bd6-68c559c443fa None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:00.742959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c4d6cb66-922a-4994-8bd6-68c559c443fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:00.804720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1217/2447] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:00.813124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eea60-e029-4d23-8021-5f1adaf46186 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:00.813440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eea60-e029-4d23-8021-5f1adaf46186 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:00.813766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1eea60-e029-4d23-8021-5f1adaf46186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:00.834845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6c1eea60-e029-4d23-8021-5f1adaf46186 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:00.836130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6c1eea60-e029-4d23-8021-5f1adaf46186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:00.844357 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1231/2448] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:00.848937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6cb66-922a-4994-8bd6-68c559c443fa None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:00.858704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6cb66-922a-4994-8bd6-68c559c443fa None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:00.864330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:00.864939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:00.865219 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:00.865527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:00.866183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:39:00.866500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:00.868082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:00.880815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1218/2449] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:39:00.885373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eea60-e029-4d23-8021-5f1adaf46186 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:00.895451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1eea60-e029-4d23-8021-5f1adaf46186 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:00.900517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a60b14-210f-4370-b6aa-0ccf81e0bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:00.901148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a60b14-210f-4370-b6aa-0ccf81e0bb78 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:00.901414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a60b14-210f-4370-b6aa-0ccf81e0bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:00.901724 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a60b14-210f-4370-b6aa-0ccf81e0bb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:00.902375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a60b14-210f-4370-b6aa-0ccf81e0bb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:00.904289 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a60b14-210f-4370-b6aa-0ccf81e0bb78 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:00.919960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:00.938333 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1232/2450] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:00 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:00.940177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:00.950428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8603a-157a-4473-87b2-1788f43f0661 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:00.975900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:01.012852 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5b7b3-e52a-4d6c-98b1-bafa4a21f456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.013501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5b7b3-e52a-4d6c-98b1-bafa4a21f456 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.013805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5b7b3-e52a-4d6c-98b1-bafa4a21f456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.014116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5b7b3-e52a-4d6c-98b1-bafa4a21f456 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.014757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5b7b3-e52a-4d6c-98b1-bafa4a21f456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.016669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5b7b3-e52a-4d6c-98b1-bafa4a21f456 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.055760 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1219/2451] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:00 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:01.056224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a60b14-210f-4370-b6aa-0ccf81e0bb78 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.069840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a60b14-210f-4370-b6aa-0ccf81e0bb78 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.074833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6fd45-11ad-44f1-9927-9a664aa102a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.075517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fd45-11ad-44f1-9927-9a664aa102a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.075871 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fd45-11ad-44f1-9927-9a664aa102a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.076216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6fd45-11ad-44f1-9927-9a664aa102a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.076908 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6fd45-11ad-44f1-9927-9a664aa102a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.078569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6fd45-11ad-44f1-9927-9a664aa102a0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.090012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1233/2452] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:01.094328 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5b7b3-e52a-4d6c-98b1-bafa4a21f456 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.105182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5b7b3-e52a-4d6c-98b1-bafa4a21f456 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.109825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bc43f-3443-470c-af1b-50e00ff67ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.110438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc43f-3443-470c-af1b-50e00ff67ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.111271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc43f-3443-470c-af1b-50e00ff67ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.111682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bc43f-3443-470c-af1b-50e00ff67ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.112336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bc43f-3443-470c-af1b-50e00ff67ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.113976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61bc43f-3443-470c-af1b-50e00ff67ea2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.124875 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1220/2453] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:01.129238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6fd45-11ad-44f1-9927-9a664aa102a0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.139663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6fd45-11ad-44f1-9927-9a664aa102a0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.145189 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c745f9c-27bd-4be7-a4cc-ce743011730b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.146048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c745f9c-27bd-4be7-a4cc-ce743011730b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.146407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c745f9c-27bd-4be7-a4cc-ce743011730b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.146753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c745f9c-27bd-4be7-a4cc-ce743011730b admin admin] PATH_INFO: `/v3/projects/b8e1a0e2742f4a13b202daaa7b046235/users/44af9b1cb63c457fba9d2fd7dee440c0/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.162240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c745f9c-27bd-4be7-a4cc-ce743011730b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e1a0e2742f4a13b202daaa7b046235, user_id=44af9b1cb63c457fba9d2fd7dee440c0, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.165358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c745f9c-27bd-4be7-a4cc-ce743011730b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.180209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1234/2454] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:01 2022] PUT /identity/v3/projects/b8e1a0e2742f4a13b202daaa7b046235/users/44af9b1cb63c457fba9d2fd7dee440c0/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:01.191123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc94ab-07b9-4687-ac88-410e81b4122b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.191421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc94ab-07b9-4687-ac88-410e81b4122b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.191699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc94ab-07b9-4687-ac88-410e81b4122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.223826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-89fc94ab-07b9-4687-ac88-410e81b4122b None None] MFA Rules not processed for user `44af9b1cb63c457fba9d2fd7dee440c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:01.225068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-89fc94ab-07b9-4687-ac88-410e81b4122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:01.293349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1221/2455] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:01.297934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c745f9c-27bd-4be7-a4cc-ce743011730b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.364123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c745f9c-27bd-4be7-a4cc-ce743011730b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.369291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224f29b-eec8-42a3-9a6d-81084f6ab792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.370008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224f29b-eec8-42a3-9a6d-81084f6ab792 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.370363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224f29b-eec8-42a3-9a6d-81084f6ab792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.370728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224f29b-eec8-42a3-9a6d-81084f6ab792 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.371396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224f29b-eec8-42a3-9a6d-81084f6ab792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.373366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224f29b-eec8-42a3-9a6d-81084f6ab792 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.403017 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1235/2456] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:01 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:39:01.407081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc94ab-07b9-4687-ac88-410e81b4122b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.418114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc94ab-07b9-4687-ac88-410e81b4122b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.474800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ceb72-ee2a-4f36-83a5-dbfcddb07c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.475417 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ceb72-ee2a-4f36-83a5-dbfcddb07c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.475695 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ceb72-ee2a-4f36-83a5-dbfcddb07c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.475968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ceb72-ee2a-4f36-83a5-dbfcddb07c83 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.476610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ceb72-ee2a-4f36-83a5-dbfcddb07c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.478616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ceb72-ee2a-4f36-83a5-dbfcddb07c83 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.501031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1222/2457] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:01 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:01.505567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224f29b-eec8-42a3-9a6d-81084f6ab792 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.516562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224f29b-eec8-42a3-9a6d-81084f6ab792 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.522269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddbcc1-c9c9-4235-8c12-583a3da19dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.522905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddbcc1-c9c9-4235-8c12-583a3da19dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.523158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddbcc1-c9c9-4235-8c12-583a3da19dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.523434 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddbcc1-c9c9-4235-8c12-583a3da19dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.524087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddbcc1-c9c9-4235-8c12-583a3da19dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.525737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddbcc1-c9c9-4235-8c12-583a3da19dbd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.538045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1236/2458] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:01.542059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ceb72-ee2a-4f36-83a5-dbfcddb07c83 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.553073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ceb72-ee2a-4f36-83a5-dbfcddb07c83 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.557969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbe08a-494a-4760-9690-f4432e0ec395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.558637 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbe08a-494a-4760-9690-f4432e0ec395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.558921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbe08a-494a-4760-9690-f4432e0ec395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.559198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbe08a-494a-4760-9690-f4432e0ec395 admin admin] PATH_INFO: `/v3/projects/36054bb58c1f4437a268a17941195e09/users/30ac4f914d82490b9e00f4644296a5fc/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.575696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbe08a-494a-4760-9690-f4432e0ec395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36054bb58c1f4437a268a17941195e09, user_id=30ac4f914d82490b9e00f4644296a5fc, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.578444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bbe08a-494a-4760-9690-f4432e0ec395 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.591673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1223/2459] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:01 2022] PUT /identity/v3/projects/36054bb58c1f4437a268a17941195e09/users/30ac4f914d82490b9e00f4644296a5fc/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:01.595927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddbcc1-c9c9-4235-8c12-583a3da19dbd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.609386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddbcc1-c9c9-4235-8c12-583a3da19dbd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.671103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b5741-9269-4441-8a0a-2636e35e451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.671770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5741-9269-4441-8a0a-2636e35e451b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.672099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5741-9269-4441-8a0a-2636e35e451b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.672459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b5741-9269-4441-8a0a-2636e35e451b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.673973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b5741-9269-4441-8a0a-2636e35e451b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.676172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b5741-9269-4441-8a0a-2636e35e451b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.687405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1237/2460] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:01.692359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbe08a-494a-4760-9690-f4432e0ec395 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.699734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:01.705388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbe08a-494a-4760-9690-f4432e0ec395 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.711817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.712501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.712791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.713072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] PATH_INFO: `/v3/projects/36054bb58c1f4437a268a17941195e09/users/30ac4f914d82490b9e00f4644296a5fc/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.715209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=36054bb58c1f4437a268a17941195e09, user_id=30ac4f914d82490b9e00f4644296a5fc, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.718028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.729323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1224/2461] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:01 2022] PUT /identity/v3/projects/36054bb58c1f4437a268a17941195e09/users/30ac4f914d82490b9e00f4644296a5fc/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:01.741473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d95afe-f463-49c2-8da3-0354868a4fbd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.741800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d95afe-f463-49c2-8da3-0354868a4fbd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.742081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d95afe-f463-49c2-8da3-0354868a4fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.771467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a4d95afe-f463-49c2-8da3-0354868a4fbd None None] MFA Rules not processed for user `30ac4f914d82490b9e00f4644296a5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:01.772698 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a4d95afe-f463-49c2-8da3-0354868a4fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:01.789334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:01.843649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1238/2462] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:01.854866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:01.856708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:01.936368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d60b53-f68c-4f51-8c5d-a0fc821c4d0a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:01.940157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4630e-b783-424b-9506-cd33cf5b481a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:01.940773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4630e-b783-424b-9506-cd33cf5b481a service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:01.941056 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4630e-b783-424b-9506-cd33cf5b481a service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:01.941339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4630e-b783-424b-9506-cd33cf5b481a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:01.950344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4630e-b783-424b-9506-cd33cf5b481a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:01.951787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4630e-b783-424b-9506-cd33cf5b481a service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:01.961308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1225/2463] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:02.039246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d95afe-f463-49c2-8da3-0354868a4fbd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:02.049524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d95afe-f463-49c2-8da3-0354868a4fbd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:02.053188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:02.053820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:02.054109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:02.054389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:02.118393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:02.119860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:02.129185 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1239/2464] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:11.957106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:12.124948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=49 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:20.545289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1e53-61be-4472-bb68-94934032d922 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:20.545889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1e53-61be-4472-bb68-94934032d922 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:20.546008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1e53-61be-4472-bb68-94934032d922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:20.577208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-967c1e53-61be-4472-bb68-94934032d922 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:20.578574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-967c1e53-61be-4472-bb68-94934032d922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:20.671419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1226/2465] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:39:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:20.744850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:20.746439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:20.831689 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d03bf-ec50-4e2c-bfb0-dae87b7b787e service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:20.836745 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:20.837362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:20.837681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:20.837962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:20.846972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:20.848566 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:20.856612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1240/2466] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:39:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 03 13:39:30.662527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=32 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:30.853395 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=32 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:34.871598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c1e53-61be-4472-bb68-94934032d922 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:34.873672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-967c1e53-61be-4472-bb68-94934032d922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:34.979089 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c1e53-61be-4472-bb68-94934032d922 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:34.982713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab80e6-2e25-43b8-a773-16aa2deaadc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:34.983330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab80e6-2e25-43b8-a773-16aa2deaadc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:34.983612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab80e6-2e25-43b8-a773-16aa2deaadc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:34.983880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab80e6-2e25-43b8-a773-16aa2deaadc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:34.985815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d9ab80e6-2e25-43b8-a773-16aa2deaadc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:35.064036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab80e6-2e25-43b8-a773-16aa2deaadc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:35.066467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab80e6-2e25-43b8-a773-16aa2deaadc9 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:35.080061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1227/2467] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:39:36.543965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:36.546981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:36.570902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11912b94-0bb5-42ea-a28b-d2c3388752b6 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:36.575936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937104da-8394-4df9-b274-6c311e53c8aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:36.577104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937104da-8394-4df9-b274-6c311e53c8aa service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:36.577623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937104da-8394-4df9-b274-6c311e53c8aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:36.578166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937104da-8394-4df9-b274-6c311e53c8aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:36.580375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-937104da-8394-4df9-b274-6c311e53c8aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:36.594685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937104da-8394-4df9-b274-6c311e53c8aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:36.596149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937104da-8394-4df9-b274-6c311e53c8aa service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:36.607251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1241/2468] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:36.774682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82345e7-f4a5-4343-9c60-b8bf23fa3cb3 None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:36.775222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82345e7-f4a5-4343-9c60-b8bf23fa3cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:36.775222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82345e7-f4a5-4343-9c60-b8bf23fa3cb3 None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:36.776887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1228/2469] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:39:36.782881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83afa8-8103-4762-8444-8aac927557fb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:36.783181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83afa8-8103-4762-8444-8aac927557fb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:36.783461 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83afa8-8103-4762-8444-8aac927557fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:36.821503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1f83afa8-8103-4762-8444-8aac927557fb None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:36.822981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1f83afa8-8103-4762-8444-8aac927557fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:36.909931 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1242/2470] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:39:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:36.923780 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82345e7-f4a5-4343-9c60-b8bf23fa3cb3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:36.926172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c82345e7-f4a5-4343-9c60-b8bf23fa3cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:37.012917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82345e7-f4a5-4343-9c60-b8bf23fa3cb3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:37.019368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59412e26-1b22-478e-a2b0-f3df663d3511 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '464190e0ccaf4f82942f518df1bf6c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:37.020097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59412e26-1b22-478e-a2b0-f3df663d3511 service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:37.020560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59412e26-1b22-478e-a2b0-f3df663d3511 service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:37.021022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59412e26-1b22-478e-a2b0-f3df663d3511 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:37.032220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59412e26-1b22-478e-a2b0-f3df663d3511 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:37.033689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59412e26-1b22-478e-a2b0-f3df663d3511 service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:37.046954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1229/2471] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:37.127408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf04df-2557-46f8-a1ba-ef105008ccb8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:37.128328 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf04df-2557-46f8-a1ba-ef105008ccb8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:37.128616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf04df-2557-46f8-a1ba-ef105008ccb8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:37.130353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1243/2472] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:39:37.135422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe8286-ea61-4ea3-a85a-e859e322762e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:37.135710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe8286-ea61-4ea3-a85a-e859e322762e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:37.136037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe8286-ea61-4ea3-a85a-e859e322762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:37.164456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7dbe8286-ea61-4ea3-a85a-e859e322762e None None] MFA Rules not processed for user `cf865cb3c7fc432884e4e308c397c83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:37.166190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7dbe8286-ea61-4ea3-a85a-e859e322762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:37.173764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1230/2473] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:39:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:37.188671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf04df-2557-46f8-a1ba-ef105008ccb8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:37.203531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf04df-2557-46f8-a1ba-ef105008ccb8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:37.208914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6928c5-a66a-44e4-a1fa-a8c37b4df164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:37.209746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6928c5-a66a-44e4-a1fa-a8c37b4df164 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:37.210248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6928c5-a66a-44e4-a1fa-a8c37b4df164 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:37.210913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6928c5-a66a-44e4-a1fa-a8c37b4df164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:37.220753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6928c5-a66a-44e4-a1fa-a8c37b4df164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:37.223367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6928c5-a66a-44e4-a1fa-a8c37b4df164 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:37.234099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1244/2474] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:37.330662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aa1f4-1c2c-406c-a7cc-b7e1e3d2459e None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:37.331283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aa1f4-1c2c-406c-a7cc-b7e1e3d2459e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:37.331953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aa1f4-1c2c-406c-a7cc-b7e1e3d2459e None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:37.333967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1231/2475] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:39:37.346901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3e9d7-9507-4f7a-8843-7ec49caaba95 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:37.347533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3e9d7-9507-4f7a-8843-7ec49caaba95 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:37.348252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3e9d7-9507-4f7a-8843-7ec49caaba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:37.370582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-74f3e9d7-9507-4f7a-8843-7ec49caaba95 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:37.371852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-74f3e9d7-9507-4f7a-8843-7ec49caaba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:37.383182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1245/2476] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:39:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:37.395705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aa1f4-1c2c-406c-a7cc-b7e1e3d2459e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:37.409348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aa1f4-1c2c-406c-a7cc-b7e1e3d2459e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:37.413532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:37.414216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:37.414499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:37.414908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:37.427783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:37.429434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:37.443125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1232/2477] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:39:39.080179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3e9d7-9507-4f7a-8843-7ec49caaba95 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:39.081812 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-74f3e9d7-9507-4f7a-8843-7ec49caaba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:39.148469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:39.152326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:39.169306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3e9d7-9507-4f7a-8843-7ec49caaba95 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:39.172172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9be1ee-fcee-4957-a430-f51afa09afbb service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:39.175252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:39.176070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:39.176361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:39.176638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:39.178440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:39.178587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:39.179399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:39.179713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:39.179984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] PATH_INFO: `/v3/users/4471a60040ca42649f2718fd46659e31` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:39.181108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4471a60040ca42649f2718fd46659e31)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:39.183061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:39.191545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:39.193324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:39.204898 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:39.205267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1246/2478] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:39.268220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4471a60040ca42649f2718fd46659e31'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.270220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4471a60040ca42649f2718fd46659e31'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.271241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4471a60040ca42649f2718fd46659e31'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.281218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4471a60040ca42649f2718fd46659e31'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.297184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-424c40ac-0aec-4dcc-9146-e46d572d3833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4471a60040ca42649f2718fd46659e31'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.320520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1233/2479] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:39 2022] DELETE /identity/v3/users/4471a60040ca42649f2718fd46659e31 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:39.631109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:39.642604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f96e-c12a-4e48-a3b1-c8543a0a5089 service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:39.674383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:39.708610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:39.709060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:39.709351 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:39.709659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] PATH_INFO: `/v3/projects/1432ad4a21624574b8caeaf10f2f8339` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:39.710930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1432ad4a21624574b8caeaf10f2f8339)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:39.712902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:39.771365 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1432ad4a21624574b8caeaf10f2f8339', 'request_id': 'req-244628d8-82b4-4565-bd3f-13377c05ec47'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.777542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1432ad4a21624574b8caeaf10f2f8339', 'request_id': 'req-244628d8-82b4-4565-bd3f-13377c05ec47'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.787267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1432ad4a21624574b8caeaf10f2f8339', 'request_id': 'req-244628d8-82b4-4565-bd3f-13377c05ec47'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.802525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] The token cache is being invalidate because project 1432ad4a21624574b8caeaf10f2f8339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:39.802904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.803544 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-244628d8-82b4-4565-bd3f-13377c05ec47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:39.805794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1247/2480] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:39 2022] DELETE /identity/v3/projects/1432ad4a21624574b8caeaf10f2f8339 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:39.810142 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb0e11-ead4-46a6-b147-8f37d6dae5a5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:39.810431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb0e11-ead4-46a6-b147-8f37d6dae5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:39.810706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb0e11-ead4-46a6-b147-8f37d6dae5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:39.833791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-dcbb0e11-ead4-46a6-b147-8f37d6dae5a5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:39.835364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-dcbb0e11-ead4-46a6-b147-8f37d6dae5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:39.914380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1234/2481] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:39.924077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ccc5a-2a4c-4627-a436-570c056b56cf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:39.925020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ccc5a-2a4c-4627-a436-570c056b56cf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:39.925471 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ccc5a-2a4c-4627-a436-570c056b56cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:39.947290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-789ccc5a-2a4c-4627-a436-570c056b56cf None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:39.948541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-789ccc5a-2a4c-4627-a436-570c056b56cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:39.957684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1248/2482] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:39.962044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb0e11-ead4-46a6-b147-8f37d6dae5a5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:39.972815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb0e11-ead4-46a6-b147-8f37d6dae5a5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:39.977592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:39.978215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:39.978502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:39.978781 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:39.979441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:39:39.979749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:39.981388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:39.994185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1235/2483] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:39:39.999121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ccc5a-2a4c-4627-a436-570c056b56cf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:40.009242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ccc5a-2a4c-4627-a436-570c056b56cf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:40.014004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fa30f-fcc5-43b7-be7a-869e858358c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:40.014626 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fa30f-fcc5-43b7-be7a-869e858358c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:40.014913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fa30f-fcc5-43b7-be7a-869e858358c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:40.015191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fa30f-fcc5-43b7-be7a-869e858358c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:40.015850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fa30f-fcc5-43b7-be7a-869e858358c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:40.017814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fa30f-fcc5-43b7-be7a-869e858358c7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:40.050024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1249/2484] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:39 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:40.054395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:40.065831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69dfd-0a2c-42ac-b9f9-4a6aa35e15a5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:40.125631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd56095-4923-4728-9df8-bb766d120078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:40.126159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd56095-4923-4728-9df8-bb766d120078 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:40.126443 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd56095-4923-4728-9df8-bb766d120078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:40.126708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd56095-4923-4728-9df8-bb766d120078 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:40.127354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd56095-4923-4728-9df8-bb766d120078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:40.129878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd56095-4923-4728-9df8-bb766d120078 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:40.152749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1236/2485] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:40 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:40.156467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fa30f-fcc5-43b7-be7a-869e858358c7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:40.167828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fa30f-fcc5-43b7-be7a-869e858358c7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:40.172925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68474f-7766-4aae-bf73-cbaf34c18ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:40.173571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68474f-7766-4aae-bf73-cbaf34c18ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:40.173856 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68474f-7766-4aae-bf73-cbaf34c18ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:40.174130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68474f-7766-4aae-bf73-cbaf34c18ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:40.174841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68474f-7766-4aae-bf73-cbaf34c18ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:40.176457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68474f-7766-4aae-bf73-cbaf34c18ac0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:40.186982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1250/2486] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:40.191829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd56095-4923-4728-9df8-bb766d120078 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:40.203419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd56095-4923-4728-9df8-bb766d120078 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:40.209193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8bf52-e331-4b30-a94a-0c0bcddeee3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:40.209878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8bf52-e331-4b30-a94a-0c0bcddeee3a admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:40.210192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8bf52-e331-4b30-a94a-0c0bcddeee3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:40.210475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8bf52-e331-4b30-a94a-0c0bcddeee3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:40.211135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8bf52-e331-4b30-a94a-0c0bcddeee3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:40.212741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd8bf52-e331-4b30-a94a-0c0bcddeee3a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:40.218476 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:40.229698 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1237/2487] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:40.230961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68474f-7766-4aae-bf73-cbaf34c18ac0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:40.242008 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68474f-7766-4aae-bf73-cbaf34c18ac0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:40.250896 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:40.251590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:40.251878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:40.252157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] PATH_INFO: `/v3/projects/f83035ae509c47fd92f821dd8edddae5/users/83b8a68047b24d67a834b55ad9be2827/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:40.269668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f83035ae509c47fd92f821dd8edddae5, user_id=83b8a68047b24d67a834b55ad9be2827, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:40.272505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:40.287432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1251/2488] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:40 2022] PUT /identity/v3/projects/f83035ae509c47fd92f821dd8edddae5/users/83b8a68047b24d67a834b55ad9be2827/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:40.292539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa9b2-f673-450b-9807-aa7324db84d6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:40.292833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa9b2-f673-450b-9807-aa7324db84d6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:40.293382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fa9b2-f673-450b-9807-aa7324db84d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:40.315028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-420fa9b2-f673-450b-9807-aa7324db84d6 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:40.317272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-420fa9b2-f673-450b-9807-aa7324db84d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:40.394196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1238/2489] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:40.404242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:40.406359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:40.465658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cca99-840a-4a88-855e-3d7baed3c442 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:40.469358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b98ce-b1e5-4e07-9145-b0fee722001a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:40.470025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b98ce-b1e5-4e07-9145-b0fee722001a service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:40.470310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b98ce-b1e5-4e07-9145-b0fee722001a service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:40.470588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b98ce-b1e5-4e07-9145-b0fee722001a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:40.481009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b98ce-b1e5-4e07-9145-b0fee722001a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:40.482495 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b98ce-b1e5-4e07-9145-b0fee722001a service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:40.486613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:40.497615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1252/2490] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:41.279740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29f72-3cdc-4752-9583-c9319145a378 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:41.280331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29f72-3cdc-4752-9583-c9319145a378 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:41.280655 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29f72-3cdc-4752-9583-c9319145a378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:41.329312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-38b29f72-3cdc-4752-9583-c9319145a378 None None] MFA Rules not processed for user `83b8a68047b24d67a834b55ad9be2827`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:41.332570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-38b29f72-3cdc-4752-9583-c9319145a378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:41.425727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1239/2491] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:41 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:41.435069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b98ce-b1e5-4e07-9145-b0fee722001a service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:41.442995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b98ce-b1e5-4e07-9145-b0fee722001a service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:41.451406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b287d-9c08-4d3a-90b4-0222f8edacb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:41.452655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b287d-9c08-4d3a-90b4-0222f8edacb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:41.453144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b287d-9c08-4d3a-90b4-0222f8edacb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:41.453561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b287d-9c08-4d3a-90b4-0222f8edacb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:41.454764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b287d-9c08-4d3a-90b4-0222f8edacb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:41.461315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b287d-9c08-4d3a-90b4-0222f8edacb7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:41.504696 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1253/2492] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:41 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:41.509692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b29f72-3cdc-4752-9583-c9319145a378 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:41.524166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b29f72-3cdc-4752-9583-c9319145a378 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:41.610716 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a5fb0-2abd-4fd6-93fb-77a5de39838c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:41.611677 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5fb0-2abd-4fd6-93fb-77a5de39838c admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:41.613482 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5fb0-2abd-4fd6-93fb-77a5de39838c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:41.614244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5fb0-2abd-4fd6-93fb-77a5de39838c admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:41.615252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a5fb0-2abd-4fd6-93fb-77a5de39838c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:41.617857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a5fb0-2abd-4fd6-93fb-77a5de39838c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:41.647050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1240/2493] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:41 2022] POST /identity/v3/users => generated 308 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:39:41.651655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b287d-9c08-4d3a-90b4-0222f8edacb7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:41.663549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b287d-9c08-4d3a-90b4-0222f8edacb7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:41.669985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2878d9-0422-40ce-bd43-8ba7e8614056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:41.670649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2878d9-0422-40ce-bd43-8ba7e8614056 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:41.671146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2878d9-0422-40ce-bd43-8ba7e8614056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:41.671625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2878d9-0422-40ce-bd43-8ba7e8614056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:41.672470 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2878d9-0422-40ce-bd43-8ba7e8614056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:41.675118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2878d9-0422-40ce-bd43-8ba7e8614056 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:41.691813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1254/2494] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:41.696121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a5fb0-2abd-4fd6-93fb-77a5de39838c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:41.707198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a5fb0-2abd-4fd6-93fb-77a5de39838c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:41.717500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426084d-5309-45d2-ba4a-4fa07e227687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:41.717500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426084d-5309-45d2-ba4a-4fa07e227687 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:41.717500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426084d-5309-45d2-ba4a-4fa07e227687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:41.717500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426084d-5309-45d2-ba4a-4fa07e227687 admin admin] PATH_INFO: `/v3/projects/37cb913ed6274b2c99fe2d2a85eb1430/users/8d95bb09402247239c48e509c5eefda8/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:41.735544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426084d-5309-45d2-ba4a-4fa07e227687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cb913ed6274b2c99fe2d2a85eb1430, user_id=8d95bb09402247239c48e509c5eefda8, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:41.739039 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426084d-5309-45d2-ba4a-4fa07e227687 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:41.753756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:41.761177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1241/2495] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:41 2022] PUT /identity/v3/projects/37cb913ed6274b2c99fe2d2a85eb1430/users/8d95bb09402247239c48e509c5eefda8/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:41.765763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2878d9-0422-40ce-bd43-8ba7e8614056 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:41.776970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2878d9-0422-40ce-bd43-8ba7e8614056 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:41.840625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef941e-da78-46c5-86b7-139c38b7831f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:41.841246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef941e-da78-46c5-86b7-139c38b7831f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:41.841641 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef941e-da78-46c5-86b7-139c38b7831f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:41.841850 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef941e-da78-46c5-86b7-139c38b7831f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:41.842506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef941e-da78-46c5-86b7-139c38b7831f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:41.844292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef941e-da78-46c5-86b7-139c38b7831f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:41.857251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1255/2496] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:41.861549 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426084d-5309-45d2-ba4a-4fa07e227687 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:41.872631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426084d-5309-45d2-ba4a-4fa07e227687 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:41.877612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a142408-4252-4729-b506-e30232147a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:41.878381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a142408-4252-4729-b506-e30232147a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:41.878662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a142408-4252-4729-b506-e30232147a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:41.878930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a142408-4252-4729-b506-e30232147a30 admin admin] PATH_INFO: `/v3/projects/37cb913ed6274b2c99fe2d2a85eb1430/users/8d95bb09402247239c48e509c5eefda8/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:41.880862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a142408-4252-4729-b506-e30232147a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cb913ed6274b2c99fe2d2a85eb1430, user_id=8d95bb09402247239c48e509c5eefda8, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:41.883679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a142408-4252-4729-b506-e30232147a30 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:41.894475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1242/2497] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:41 2022] PUT /identity/v3/projects/37cb913ed6274b2c99fe2d2a85eb1430/users/8d95bb09402247239c48e509c5eefda8/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:42.728384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea2ca4-6ebf-43c6-8c91-f0e90575311d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:42.729098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea2ca4-6ebf-43c6-8c91-f0e90575311d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:42.729098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea2ca4-6ebf-43c6-8c91-f0e90575311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:42.760830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-faea2ca4-6ebf-43c6-8c91-f0e90575311d None None] MFA Rules not processed for user `8d95bb09402247239c48e509c5eefda8`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:42.762568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-faea2ca4-6ebf-43c6-8c91-f0e90575311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:42.884853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1256/2498] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:43.175143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038352-9668-415f-b778-74a6df0e564d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:43.175849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038352-9668-415f-b778-74a6df0e564d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:43.176262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038352-9668-415f-b778-74a6df0e564d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:43.200711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-70038352-9668-415f-b778-74a6df0e564d None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:43.202381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-70038352-9668-415f-b778-74a6df0e564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:43.265718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1243/2499] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:39:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:43.278627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea2ca4-6ebf-43c6-8c91-f0e90575311d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:43.280089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-faea2ca4-6ebf-43c6-8c91-f0e90575311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:43.338785 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:43.370139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea2ca4-6ebf-43c6-8c91-f0e90575311d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:43.376109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876834ab-285a-439a-bd41-ec5e0f9d09dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:43.377041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876834ab-285a-439a-bd41-ec5e0f9d09dd service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:43.377599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876834ab-285a-439a-bd41-ec5e0f9d09dd service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:43.377946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876834ab-285a-439a-bd41-ec5e0f9d09dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:43.391599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876834ab-285a-439a-bd41-ec5e0f9d09dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:43.393535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876834ab-285a-439a-bd41-ec5e0f9d09dd service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:43.403257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1257/2500] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:39:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:39:43.508517 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70038352-9668-415f-b778-74a6df0e564d None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:43.511839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-70038352-9668-415f-b778-74a6df0e564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:43.621271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70038352-9668-415f-b778-74a6df0e564d None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:43.629623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:43.630592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:43.631091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:43.631658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:43.652464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:43.684268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:43.689967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:43.716860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1244/2501] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:44.350986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876834ab-285a-439a-bd41-ec5e0f9d09dd service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:44.479115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876834ab-285a-439a-bd41-ec5e0f9d09dd service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:44.490725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:44.492376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:44.493055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:44.493615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] PATH_INFO: `/v3/users/83b8a68047b24d67a834b55ad9be2827` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:44.497103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b8a68047b24d67a834b55ad9be2827)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:44.504930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:44.605920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b8a68047b24d67a834b55ad9be2827'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:44.625686 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b8a68047b24d67a834b55ad9be2827'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:44.626938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b8a68047b24d67a834b55ad9be2827'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:44.642084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b8a68047b24d67a834b55ad9be2827'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:44.656535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b8a68047b24d67a834b55ad9be2827'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:44.678119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1258/2502] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:44 2022] DELETE /identity/v3/users/83b8a68047b24d67a834b55ad9be2827 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:44.973197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:44.974871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:45.004704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efe6ae-d78b-4313-aaad-ce3f67c6d701 service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:45.011395 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:45.013379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:45.171720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:45.176936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:45.178818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:45.179586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:45.180533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] PATH_INFO: `/v3/users/44af9b1cb63c457fba9d2fd7dee440c0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:45.183196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44af9b1cb63c457fba9d2fd7dee440c0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:45.189196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:45.192475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda88370-8350-4af6-9502-f74d4233ce5a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:45.211608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:45.212277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:45.212590 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:45.212834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] PATH_INFO: `/v3/projects/f83035ae509c47fd92f821dd8edddae5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:45.214046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f83035ae509c47fd92f821dd8edddae5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:45.216019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:45.254211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:45.321726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44af9b1cb63c457fba9d2fd7dee440c0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.322890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44af9b1cb63c457fba9d2fd7dee440c0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.324068 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44af9b1cb63c457fba9d2fd7dee440c0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.324371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f83035ae509c47fd92f821dd8edddae5', 'request_id': 'req-1ba7a842-ec97-49b2-aab6-a7bae0819f86'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.344894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44af9b1cb63c457fba9d2fd7dee440c0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.350896 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f83035ae509c47fd92f821dd8edddae5', 'request_id': 'req-1ba7a842-ec97-49b2-aab6-a7bae0819f86'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.362375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f83035ae509c47fd92f821dd8edddae5', 'request_id': 'req-1ba7a842-ec97-49b2-aab6-a7bae0819f86'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.380375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] The token cache is being invalidate because project f83035ae509c47fd92f821dd8edddae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:45.381129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.382121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.388452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1259/2503] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:45 2022] DELETE /identity/v3/projects/f83035ae509c47fd92f821dd8edddae5 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:45.388706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b13e9cbd-a7b2-4746-9bf6-23b33abbf455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44af9b1cb63c457fba9d2fd7dee440c0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.390494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:45.393672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:45.416809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1245/2504] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:44 2022] DELETE /identity/v3/users/44af9b1cb63c457fba9d2fd7dee440c0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:45.419062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c37530-244e-4d96-a38e-ef58254f8f61 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:45.419468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c37530-244e-4d96-a38e-ef58254f8f61 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:45.420080 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c37530-244e-4d96-a38e-ef58254f8f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:45.445704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-46c37530-244e-4d96-a38e-ef58254f8f61 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:45.447589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-46c37530-244e-4d96-a38e-ef58254f8f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:45.522557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7a842-ec97-49b2-aab6-a7bae0819f86 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:45.576363 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1246/2505] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:45.592497 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c37530-244e-4d96-a38e-ef58254f8f61 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:45.594704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-46c37530-244e-4d96-a38e-ef58254f8f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:45.636049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:45.637221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:45.637810 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:45.638287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] PATH_INFO: `/v3/users/8d95bb09402247239c48e509c5eefda8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:45.640263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d95bb09402247239c48e509c5eefda8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:45.644719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:45.682344 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c37530-244e-4d96-a38e-ef58254f8f61 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:45.687176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839fb71-0387-44a2-8caa-ca193a32634b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:45.687797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fb71-0387-44a2-8caa-ca193a32634b service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:45.688122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fb71-0387-44a2-8caa-ca193a32634b service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:45.688745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fb71-0387-44a2-8caa-ca193a32634b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:45.701740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839fb71-0387-44a2-8caa-ca193a32634b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:45.703299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839fb71-0387-44a2-8caa-ca193a32634b service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:45.716711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:45.730480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1247/2506] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:39:45.732381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d95bb09402247239c48e509c5eefda8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.760915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d95bb09402247239c48e509c5eefda8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.763070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d95bb09402247239c48e509c5eefda8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.786659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d95bb09402247239c48e509c5eefda8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.803516 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d95bb09402247239c48e509c5eefda8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:45.824615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1260/2507] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:45 2022] DELETE /identity/v3/users/8d95bb09402247239c48e509c5eefda8 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:46.218275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839fb71-0387-44a2-8caa-ca193a32634b service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:46.242227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839fb71-0387-44a2-8caa-ca193a32634b service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:46.254902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:46.257581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:46.360606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:46.414691 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:46.417171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:46.417811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:46.418291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] PATH_INFO: `/v3/projects/37cb913ed6274b2c99fe2d2a85eb1430` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:46.420654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37cb913ed6274b2c99fe2d2a85eb1430)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:46.425254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:46.447853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c10d5b-edec-4fb9-80e9-0d15b8c6a39e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:46.464018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:46.464836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:46.465229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:46.465777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] PATH_INFO: `/v3/projects/b8e1a0e2742f4a13b202daaa7b046235` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:46.467337 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e1a0e2742f4a13b202daaa7b046235)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:46.472457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:46.542457 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37cb913ed6274b2c99fe2d2a85eb1430', 'request_id': 'req-75889bad-cd3e-4421-85bd-811db73c4a92'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.552428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37cb913ed6274b2c99fe2d2a85eb1430', 'request_id': 'req-75889bad-cd3e-4421-85bd-811db73c4a92'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.562203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37cb913ed6274b2c99fe2d2a85eb1430', 'request_id': 'req-75889bad-cd3e-4421-85bd-811db73c4a92'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.568484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e1a0e2742f4a13b202daaa7b046235', 'request_id': 'req-36a9a597-7f84-4bd8-bb09-5367d187601e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.579408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e1a0e2742f4a13b202daaa7b046235', 'request_id': 'req-36a9a597-7f84-4bd8-bb09-5367d187601e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.586719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] The token cache is being invalidate because project 37cb913ed6274b2c99fe2d2a85eb1430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:46.587291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.588066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-75889bad-cd3e-4421-85bd-811db73c4a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.589906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1248/2508] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:46 2022] DELETE /identity/v3/projects/37cb913ed6274b2c99fe2d2a85eb1430 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:46.592117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e1a0e2742f4a13b202daaa7b046235', 'request_id': 'req-36a9a597-7f84-4bd8-bb09-5367d187601e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.595346 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2924ea5-76c8-42d0-a95c-736ce30fb9f3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:46.595908 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2924ea5-76c8-42d0-a95c-736ce30fb9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:46.596391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2924ea5-76c8-42d0-a95c-736ce30fb9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:46.621533 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a2924ea5-76c8-42d0-a95c-736ce30fb9f3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:46.622785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a2924ea5-76c8-42d0-a95c-736ce30fb9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:46.632700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] The token cache is being invalidate because project b8e1a0e2742f4a13b202daaa7b046235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:46.633043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.634139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.637490 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1261/2509] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:46 2022] DELETE /identity/v3/projects/b8e1a0e2742f4a13b202daaa7b046235 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:46.641257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:46.642903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:46.707653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1249/2510] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:46.709323 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7452e81-e511-4324-b456-a676d331e497 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:46.709711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7452e81-e511-4324-b456-a676d331e497 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:46.710063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7452e81-e511-4324-b456-a676d331e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:46.715859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:46.735393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f7452e81-e511-4324-b456-a676d331e497 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:46.736624 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f7452e81-e511-4324-b456-a676d331e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:46.752022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1250/2511] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:46.753144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9a597-7f84-4bd8-bb09-5367d187601e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:46.753588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7452e81-e511-4324-b456-a676d331e497 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:46.762361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:46.763589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:46.764039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:46.764491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] PATH_INFO: `/v3/users/30ac4f914d82490b9e00f4644296a5fc` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:46.765379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7452e81-e511-4324-b456-a676d331e497 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:46.766167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ac4f914d82490b9e00f4644296a5fc)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:46.769083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:46.772551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:46.773615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:46.774087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:46.774407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:46.775138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:39:46.775444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:46.777057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:46.795293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1251/2512] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:39:46.796226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:46.810062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755f8bc-a522-4cac-96f1-2a9e50bcaddf admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:46.817620 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330d5e-1d2e-4bbf-99e5-7ac4672319f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:46.818720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330d5e-1d2e-4bbf-99e5-7ac4672319f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:46.819292 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330d5e-1d2e-4bbf-99e5-7ac4672319f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:46.819779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46330d5e-1d2e-4bbf-99e5-7ac4672319f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:46.821197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46330d5e-1d2e-4bbf-99e5-7ac4672319f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:46.824539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46330d5e-1d2e-4bbf-99e5-7ac4672319f1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:46.850494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ac4f914d82490b9e00f4644296a5fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.880001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1252/2513] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:46 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:46.881239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330d5e-1d2e-4bbf-99e5-7ac4672319f1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:46.887085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ac4f914d82490b9e00f4644296a5fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.888198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ac4f914d82490b9e00f4644296a5fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.901521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ac4f914d82490b9e00f4644296a5fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.910874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ac4f914d82490b9e00f4644296a5fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:46.925974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1262/2514] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:46 2022] DELETE /identity/v3/users/30ac4f914d82490b9e00f4644296a5fc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:46.962503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:47.055078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46330d5e-1d2e-4bbf-99e5-7ac4672319f1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:47.261099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:47.261830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.265498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.265808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.266481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:47.274847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:47.337152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1253/2515] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:46 2022] POST /identity/v3/users => generated 290 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:47.341568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:47.341568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:47.417432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f840d03-77f0-4bc2-83ab-15078e040061 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:47.452562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a20a0-a3d8-4988-b6e6-599f3f49fb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:47.460723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a20a0-a3d8-4988-b6e6-599f3f49fb15 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.461301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a20a0-a3d8-4988-b6e6-599f3f49fb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.462936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a20a0-a3d8-4988-b6e6-599f3f49fb15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.469679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a20a0-a3d8-4988-b6e6-599f3f49fb15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:47.474479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a20a0-a3d8-4988-b6e6-599f3f49fb15 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:47.497931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:47.500651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:47.515615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1263/2516] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:47.518240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a20a0-a3d8-4988-b6e6-599f3f49fb15 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:47.537234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f18045-bc0b-41d7-a4b7-d0eea8dca3dd admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:47.543311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:47.544055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.544592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.544948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] PATH_INFO: `/v3/projects/36054bb58c1f4437a268a17941195e09` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.546213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36054bb58c1f4437a268a17941195e09)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:47.549985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:47.554231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a20a0-a3d8-4988-b6e6-599f3f49fb15 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:47.560032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae18ee-821b-42ac-90d6-6c3b3cdd3784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:47.560433 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae18ee-821b-42ac-90d6-6c3b3cdd3784 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.561090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae18ee-821b-42ac-90d6-6c3b3cdd3784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.561090 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbae18ee-821b-42ac-90d6-6c3b3cdd3784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.561688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae18ee-821b-42ac-90d6-6c3b3cdd3784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:47.563563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbae18ee-821b-42ac-90d6-6c3b3cdd3784 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:47.583275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1264/2517] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:47.588101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae18ee-821b-42ac-90d6-6c3b3cdd3784 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:47.603147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbae18ee-821b-42ac-90d6-6c3b3cdd3784 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:47.625680 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36054bb58c1f4437a268a17941195e09', 'request_id': 'req-030f0d33-c5a7-4af3-81e1-10292debbe11'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:47.634851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36054bb58c1f4437a268a17941195e09', 'request_id': 'req-030f0d33-c5a7-4af3-81e1-10292debbe11'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:47.645354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36054bb58c1f4437a268a17941195e09', 'request_id': 'req-030f0d33-c5a7-4af3-81e1-10292debbe11'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:47.665681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] The token cache is being invalidate because project 36054bb58c1f4437a268a17941195e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:47.666032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:47.666744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-030f0d33-c5a7-4af3-81e1-10292debbe11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:47.672306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1254/2518] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:47 2022] DELETE /identity/v3/projects/36054bb58c1f4437a268a17941195e09 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:47.675049 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0cbe-c58a-4a13-b3f7-61c641ed2785 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.675470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0cbe-c58a-4a13-b3f7-61c641ed2785 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.675894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0cbe-c58a-4a13-b3f7-61c641ed2785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.688637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b249bd-95ce-4354-969a-0bc6f8d4de09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:47.689488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b249bd-95ce-4354-969a-0bc6f8d4de09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.689787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b249bd-95ce-4354-969a-0bc6f8d4de09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.690618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b249bd-95ce-4354-969a-0bc6f8d4de09 admin admin] PATH_INFO: `/v3/projects/f13b3c0bb11e4c6c9f3064165847254c/users/c7bee406f0e241c7a14adcec2c0600d9/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.701616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-325f0cbe-c58a-4a13-b3f7-61c641ed2785 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:47.703070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-325f0cbe-c58a-4a13-b3f7-61c641ed2785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:47.720846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1255/2519] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:47.721100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b249bd-95ce-4354-969a-0bc6f8d4de09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13b3c0bb11e4c6c9f3064165847254c, user_id=c7bee406f0e241c7a14adcec2c0600d9, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:47.721100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b249bd-95ce-4354-969a-0bc6f8d4de09 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:47.729222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcedff-9337-4034-8d7d-eba75343aafb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.729623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcedff-9337-4034-8d7d-eba75343aafb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.730228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcedff-9337-4034-8d7d-eba75343aafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.746316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1265/2520] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:47 2022] PUT /identity/v3/projects/f13b3c0bb11e4c6c9f3064165847254c/users/c7bee406f0e241c7a14adcec2c0600d9/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:47.760382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fad709-042e-4828-bb9b-5dc885eb1ad4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.760884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fad709-042e-4828-bb9b-5dc885eb1ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.763362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fad709-042e-4828-bb9b-5dc885eb1ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.764242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-06bcedff-9337-4034-8d7d-eba75343aafb None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:47.765853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-06bcedff-9337-4034-8d7d-eba75343aafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:47.830946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:47.839151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-08fad709-042e-4828-bb9b-5dc885eb1ad4 None None] MFA Rules not processed for user `c7bee406f0e241c7a14adcec2c0600d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:47.842539 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-08fad709-042e-4828-bb9b-5dc885eb1ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:47.860848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1256/2521] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:47.862893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:47.866946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcedff-9337-4034-8d7d-eba75343aafb None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:47.877322 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcedff-9337-4034-8d7d-eba75343aafb None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:47.890893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:47.890893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.890893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.890893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.891450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:39:47.891450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:47.891450 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:47.929643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1257/2522] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:39:47.933815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:47.952251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77417182-369e-4b5b-8129-f46b16ae2dac admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:47.959026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72ca5b-1bf9-4bb7-930a-e2986a51be3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:47.959811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ca5b-1bf9-4bb7-930a-e2986a51be3a admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:47.960206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ca5b-1bf9-4bb7-930a-e2986a51be3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:47.960614 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ca5b-1bf9-4bb7-930a-e2986a51be3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:47.961594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72ca5b-1bf9-4bb7-930a-e2986a51be3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:47.964340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72ca5b-1bf9-4bb7-930a-e2986a51be3a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.007372 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1258/2523] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:47 2022] POST /identity/v3/projects => generated 363 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:48.008416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72ca5b-1bf9-4bb7-930a-e2986a51be3a admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.009695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1266/2524] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:48.014022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fad709-042e-4828-bb9b-5dc885eb1ad4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.015437 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-08fad709-042e-4828-bb9b-5dc885eb1ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:48.036781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fad709-042e-4828-bb9b-5dc885eb1ad4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.095606 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72ca5b-1bf9-4bb7-930a-e2986a51be3a admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.101706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea726885-8fbd-4ea4-8669-8b2ff8cf891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.102473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726885-8fbd-4ea4-8669-8b2ff8cf891b admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.102862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726885-8fbd-4ea4-8669-8b2ff8cf891b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.103387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726885-8fbd-4ea4-8669-8b2ff8cf891b admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.104134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea726885-8fbd-4ea4-8669-8b2ff8cf891b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.106225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea726885-8fbd-4ea4-8669-8b2ff8cf891b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.119445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61f973-374b-4bce-a442-8ab839448ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.120704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61f973-374b-4bce-a442-8ab839448ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.121263 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61f973-374b-4bce-a442-8ab839448ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.121881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61f973-374b-4bce-a442-8ab839448ae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.122700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61f973-374b-4bce-a442-8ab839448ae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.125287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61f973-374b-4bce-a442-8ab839448ae0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.139308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1259/2525] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:48 2022] POST /identity/v3/users => generated 282 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:48.145528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea726885-8fbd-4ea4-8669-8b2ff8cf891b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.164926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea726885-8fbd-4ea4-8669-8b2ff8cf891b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.170622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1267/2526] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:48 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:39:48.170845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d75dda-3b43-4889-a36b-c8a3739c0a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.171408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d75dda-3b43-4889-a36b-c8a3739c0a95 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.171825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d75dda-3b43-4889-a36b-c8a3739c0a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.172121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d75dda-3b43-4889-a36b-c8a3739c0a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.172790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d75dda-3b43-4889-a36b-c8a3739c0a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.174803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d75dda-3b43-4889-a36b-c8a3739c0a95 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.176042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61f973-374b-4bce-a442-8ab839448ae0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.199375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1260/2527] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:48.200441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d75dda-3b43-4889-a36b-c8a3739c0a95 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.215633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d75dda-3b43-4889-a36b-c8a3739c0a95 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.272881 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61f973-374b-4bce-a442-8ab839448ae0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.278602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac15b34-e38d-46e6-8d2e-2a1bafa43f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.279256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15b34-e38d-46e6-8d2e-2a1bafa43f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.279540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15b34-e38d-46e6-8d2e-2a1bafa43f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.279816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15b34-e38d-46e6-8d2e-2a1bafa43f43 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.280456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac15b34-e38d-46e6-8d2e-2a1bafa43f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.282526 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac15b34-e38d-46e6-8d2e-2a1bafa43f43 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.303436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aab112-1130-4bfd-bf77-a197f50a0408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.304448 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aab112-1130-4bfd-bf77-a197f50a0408 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.304871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aab112-1130-4bfd-bf77-a197f50a0408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.305210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aab112-1130-4bfd-bf77-a197f50a0408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.306207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aab112-1130-4bfd-bf77-a197f50a0408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.308226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aab112-1130-4bfd-bf77-a197f50a0408 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.321489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1268/2528] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:48 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:48.325734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac15b34-e38d-46e6-8d2e-2a1bafa43f43 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.331439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1261/2529] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:48.336093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aab112-1130-4bfd-bf77-a197f50a0408 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.350154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac15b34-e38d-46e6-8d2e-2a1bafa43f43 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.355558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aab112-1130-4bfd-bf77-a197f50a0408 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.357440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abc55e-81c9-493a-bae9-7d40868015fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.358199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abc55e-81c9-493a-bae9-7d40868015fd admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.358559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abc55e-81c9-493a-bae9-7d40868015fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.358940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23abc55e-81c9-493a-bae9-7d40868015fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.359718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abc55e-81c9-493a-bae9-7d40868015fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.361420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23abc55e-81c9-493a-bae9-7d40868015fd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.362066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a373608-bb87-480a-a0e1-ab91cbfab869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.363384 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373608-bb87-480a-a0e1-ab91cbfab869 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.363853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373608-bb87-480a-a0e1-ab91cbfab869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.364579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373608-bb87-480a-a0e1-ab91cbfab869 admin admin] PATH_INFO: `/v3/projects/8fb1ef095da946fb9bce9228c157a74b/users/4c3ac32d9d5642198d60ab01d96c738d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.380525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1269/2530] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:48.381628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abc55e-81c9-493a-bae9-7d40868015fd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.384719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a373608-bb87-480a-a0e1-ab91cbfab869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb1ef095da946fb9bce9228c157a74b, user_id=4c3ac32d9d5642198d60ab01d96c738d, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.387694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a373608-bb87-480a-a0e1-ab91cbfab869 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.397531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23abc55e-81c9-493a-bae9-7d40868015fd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.403340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76cac1-9d67-4457-9d8f-0249fe109b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.404100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76cac1-9d67-4457-9d8f-0249fe109b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.404466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76cac1-9d67-4457-9d8f-0249fe109b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.404752 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab76cac1-9d67-4457-9d8f-0249fe109b66 admin admin] PATH_INFO: `/v3/projects/7104850369994c3c83a5e0dcc71074b7/users/41be4bba01644a7fb22c88ad68c2481f/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.406162 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:48.414320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1262/2531] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:48 2022] PUT /identity/v3/projects/8fb1ef095da946fb9bce9228c157a74b/users/4c3ac32d9d5642198d60ab01d96c738d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:48.425744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aba3d-d568-45cc-8e66-56cf27297cf5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.426033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aba3d-d568-45cc-8e66-56cf27297cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.426317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5aba3d-d568-45cc-8e66-56cf27297cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.438172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:48.442120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76cac1-9d67-4457-9d8f-0249fe109b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7104850369994c3c83a5e0dcc71074b7, user_id=41be4bba01644a7fb22c88ad68c2481f, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.446629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab76cac1-9d67-4457-9d8f-0249fe109b66 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.463306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-fc5aba3d-d568-45cc-8e66-56cf27297cf5 None None] MFA Rules not processed for user `4c3ac32d9d5642198d60ab01d96c738d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:48.465037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-fc5aba3d-d568-45cc-8e66-56cf27297cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:48.473554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1270/2532] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:48 2022] PUT /identity/v3/projects/7104850369994c3c83a5e0dcc71074b7/users/41be4bba01644a7fb22c88ad68c2481f/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:48.477055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76cac1-9d67-4457-9d8f-0249fe109b66 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.604229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1263/2533] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:48.608373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aba3d-d568-45cc-8e66-56cf27297cf5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.647835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab76cac1-9d67-4457-9d8f-0249fe109b66 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.653680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20765e39-49e0-4bcd-9c63-80b4b6a17144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.654401 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20765e39-49e0-4bcd-9c63-80b4b6a17144 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.654762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20765e39-49e0-4bcd-9c63-80b4b6a17144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.655042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20765e39-49e0-4bcd-9c63-80b4b6a17144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.655799 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20765e39-49e0-4bcd-9c63-80b4b6a17144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.657816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20765e39-49e0-4bcd-9c63-80b4b6a17144 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.681625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1271/2534] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:48.686576 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20765e39-49e0-4bcd-9c63-80b4b6a17144 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.708391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20765e39-49e0-4bcd-9c63-80b4b6a17144 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.717292 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b71c0-7d76-482f-a630-8279bf2d944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.718833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b71c0-7d76-482f-a630-8279bf2d944f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.719187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b71c0-7d76-482f-a630-8279bf2d944f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.719543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b71c0-7d76-482f-a630-8279bf2d944f admin admin] PATH_INFO: `/v3/projects/7104850369994c3c83a5e0dcc71074b7/users/41be4bba01644a7fb22c88ad68c2481f/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.728993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b71c0-7d76-482f-a630-8279bf2d944f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7104850369994c3c83a5e0dcc71074b7, user_id=41be4bba01644a7fb22c88ad68c2481f, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.740767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b71c0-7d76-482f-a630-8279bf2d944f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.751452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5aba3d-d568-45cc-8e66-56cf27297cf5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:48.764231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1272/2535] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:48 2022] PUT /identity/v3/projects/7104850369994c3c83a5e0dcc71074b7/users/41be4bba01644a7fb22c88ad68c2481f/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:48.777823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09c04-1794-4663-97a1-3ed713e32f46 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.779518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09c04-1794-4663-97a1-3ed713e32f46 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.780159 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09c04-1794-4663-97a1-3ed713e32f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.841110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-31e09c04-1794-4663-97a1-3ed713e32f46 None None] MFA Rules not processed for user `41be4bba01644a7fb22c88ad68c2481f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:48.842577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-31e09c04-1794-4663-97a1-3ed713e32f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:48.899160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc26e-489e-4e8b-9e1f-47a12428f6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:48.899812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc26e-489e-4e8b-9e1f-47a12428f6a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:48.900106 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc26e-489e-4e8b-9e1f-47a12428f6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:48.900384 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc26e-489e-4e8b-9e1f-47a12428f6a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:48.901031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdc26e-489e-4e8b-9e1f-47a12428f6a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:48.903216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdc26e-489e-4e8b-9e1f-47a12428f6a1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:48.946798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1264/2536] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:48 2022] POST /identity/v3/projects => generated 363 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:48.950849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc26e-489e-4e8b-9e1f-47a12428f6a1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.974549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1273/2537] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:48.984977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:48.992404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e09c04-1794-4663-97a1-3ed713e32f46 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:48.999682 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-31e09c04-1794-4663-97a1-3ed713e32f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:49.070704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc26e-489e-4e8b-9e1f-47a12428f6a1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:49.095932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb044b-c323-4f30-b87a-afad21c0c133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:49.097181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb044b-c323-4f30-b87a-afad21c0c133 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:49.097488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb044b-c323-4f30-b87a-afad21c0c133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:49.097959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb044b-c323-4f30-b87a-afad21c0c133 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:49.098615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb044b-c323-4f30-b87a-afad21c0c133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:49.100587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb044b-c323-4f30-b87a-afad21c0c133 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:49.106994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e09c04-1794-4663-97a1-3ed713e32f46 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:49.126223 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:49.133977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1265/2538] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:48 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:49.137944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb044b-c323-4f30-b87a-afad21c0c133 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:49.148702 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb044b-c323-4f30-b87a-afad21c0c133 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:49.154423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b850f-e3bd-4cf0-9089-28db8c499163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:49.154423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b850f-e3bd-4cf0-9089-28db8c499163 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:49.154711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b850f-e3bd-4cf0-9089-28db8c499163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:49.154967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b850f-e3bd-4cf0-9089-28db8c499163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:49.155625 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b850f-e3bd-4cf0-9089-28db8c499163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:49.157499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b850f-e3bd-4cf0-9089-28db8c499163 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:49.173689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1266/2539] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:49.179280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370778d-4dcb-4845-acca-495b9bbd474b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:49.180065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370778d-4dcb-4845-acca-495b9bbd474b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:49.180497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370778d-4dcb-4845-acca-495b9bbd474b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:49.180619 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b850f-e3bd-4cf0-9089-28db8c499163 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:49.180956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370778d-4dcb-4845-acca-495b9bbd474b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:49.198434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b850f-e3bd-4cf0-9089-28db8c499163 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:49.220734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a5740-59e7-4c9c-917a-9ca407746138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:49.222078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a5740-59e7-4c9c-917a-9ca407746138 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:49.222365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a5740-59e7-4c9c-917a-9ca407746138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:49.222644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a5740-59e7-4c9c-917a-9ca407746138 admin admin] PATH_INFO: `/v3/projects/878fe08d009a4399877a4124667421a8/users/4a65fc0220e54175a8c55793dbbd2b0a/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:49.241291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a5740-59e7-4c9c-917a-9ca407746138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=878fe08d009a4399877a4124667421a8, user_id=4a65fc0220e54175a8c55793dbbd2b0a, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:49.244013 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a5740-59e7-4c9c-917a-9ca407746138 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:49.266339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1267/2540] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:49 2022] PUT /identity/v3/projects/878fe08d009a4399877a4124667421a8/users/4a65fc0220e54175a8c55793dbbd2b0a/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:49.270887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a5740-59e7-4c9c-917a-9ca407746138 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:49.307722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370778d-4dcb-4845-acca-495b9bbd474b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:49.310148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370778d-4dcb-4845-acca-495b9bbd474b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:49.326111 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1274/2541] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:49.394073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a5740-59e7-4c9c-917a-9ca407746138 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:49.399083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f9bdf-d902-4b0d-818f-9e37050bb3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:49.399983 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f9bdf-d902-4b0d-818f-9e37050bb3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:49.400529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f9bdf-d902-4b0d-818f-9e37050bb3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:49.400821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f9bdf-d902-4b0d-818f-9e37050bb3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:49.401514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f9bdf-d902-4b0d-818f-9e37050bb3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:49.403366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f9bdf-d902-4b0d-818f-9e37050bb3a1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:49.419118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1268/2542] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:49.423384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370778d-4dcb-4845-acca-495b9bbd474b service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:49.435632 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370778d-4dcb-4845-acca-495b9bbd474b service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:49.441671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c05a3b-ea08-471f-80d0-9538f2aae0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:49.442344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c05a3b-ea08-471f-80d0-9538f2aae0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:49.442749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c05a3b-ea08-471f-80d0-9538f2aae0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:49.443136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c05a3b-ea08-471f-80d0-9538f2aae0c4 admin admin] PATH_INFO: `/v3/projects/878fe08d009a4399877a4124667421a8/users/4a65fc0220e54175a8c55793dbbd2b0a/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:49.445260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c05a3b-ea08-471f-80d0-9538f2aae0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=878fe08d009a4399877a4124667421a8, user_id=4a65fc0220e54175a8c55793dbbd2b0a, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:49.448784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c05a3b-ea08-471f-80d0-9538f2aae0c4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:49.464359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1275/2543] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:49 2022] PUT /identity/v3/projects/878fe08d009a4399877a4124667421a8/users/4a65fc0220e54175a8c55793dbbd2b0a/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:49.477969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095a20e-740c-4e69-8189-5c477206ffaf None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:49.477969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095a20e-740c-4e69-8189-5c477206ffaf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:49.477969 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095a20e-740c-4e69-8189-5c477206ffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:49.529216 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a095a20e-740c-4e69-8189-5c477206ffaf None None] MFA Rules not processed for user `4a65fc0220e54175a8c55793dbbd2b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:49.531797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a095a20e-740c-4e69-8189-5c477206ffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:49.629367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:49.644074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1269/2544] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:49.656128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c05a3b-ea08-471f-80d0-9538f2aae0c4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:49.748388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c05a3b-ea08-471f-80d0-9538f2aae0c4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:49.754765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6298c-c3b3-41eb-8b6f-5fbfbc65a5a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:49.755457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6298c-c3b3-41eb-8b6f-5fbfbc65a5a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:49.755751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6298c-c3b3-41eb-8b6f-5fbfbc65a5a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:49.756037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6298c-c3b3-41eb-8b6f-5fbfbc65a5a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:49.770562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6298c-c3b3-41eb-8b6f-5fbfbc65a5a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:49.776943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6298c-c3b3-41eb-8b6f-5fbfbc65a5a9 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:49.792448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1276/2545] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:49.841235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095a20e-740c-4e69-8189-5c477206ffaf None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:50.006803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095a20e-740c-4e69-8189-5c477206ffaf None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:50.013100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41be4bba01644a7fb22c88ad68c2481f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7104850369994c3c83a5e0dcc71074b7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:50.014603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:50.015246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:50.015766 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:50.016934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:39:50.018861 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:50.026673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:50.046707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1270/2546] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:39:50.050991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6298c-c3b3-41eb-8b6f-5fbfbc65a5a9 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:50.065929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6298c-c3b3-41eb-8b6f-5fbfbc65a5a9 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:50.078069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ede1f-3c89-4159-a840-4c54f3a0d98a tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41be4bba01644a7fb22c88ad68c2481f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7104850369994c3c83a5e0dcc71074b7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:50.079838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ede1f-3c89-4159-a840-4c54f3a0d98a tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:50.082064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ede1f-3c89-4159-a840-4c54f3a0d98a tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:50.082560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ede1f-3c89-4159-a840-4c54f3a0d98a tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:50.083248 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ede1f-3c89-4159-a840-4c54f3a0d98a tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:50.085386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ede1f-3c89-4159-a840-4c54f3a0d98a tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:50.155219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1277/2547] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:50 2022] POST /identity/v3/projects => generated 354 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:39:50.165117 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:50.183724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b3a22-c76d-4716-bedc-c502d7005ad6 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:50.368336 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41be4bba01644a7fb22c88ad68c2481f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7104850369994c3c83a5e0dcc71074b7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:50.369073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:50.369487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:50.370327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:50.371103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:39:50.371589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:50.373501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:50.397888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1271/2548] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:39:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:39:50.447555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ede1f-3c89-4159-a840-4c54f3a0d98a tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:50.474282 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ede1f-3c89-4159-a840-4c54f3a0d98a tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:50.479968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41be4bba01644a7fb22c88ad68c2481f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7104850369994c3c83a5e0dcc71074b7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:50.485568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:50.485568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:50.485568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] PATH_INFO: `/v3/projects/063d55fdd502467ba32d24a03e2c0667` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:50.490296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Authorizing `identity:delete_project(project_id=063d55fdd502467ba32d24a03e2c0667)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:50.495288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:50.544104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:50.567011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063d55fdd502467ba32d24a03e2c0667', 'request_id': 'req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.574035 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063d55fdd502467ba32d24a03e2c0667', 'request_id': 'req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.585749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063d55fdd502467ba32d24a03e2c0667', 'request_id': 'req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.595954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:50.602416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] The token cache is being invalidate because project 063d55fdd502467ba32d24a03e2c0667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:50.603036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.603985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.606650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1278/2549] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:50 2022] DELETE /identity/v3/projects/063d55fdd502467ba32d24a03e2c0667 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:50.720245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad693b-892c-4ed4-bfb9-79c03fd523d5 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:50.757224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:50.793538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:50.794811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:50.795377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:50.795902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] PATH_INFO: `/v3/users/4c3ac32d9d5642198d60ab01d96c738d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:50.797386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3ac32d9d5642198d60ab01d96c738d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:50.800449 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:50.867624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:50.869232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:50.886086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3ac32d9d5642198d60ab01d96c738d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.887323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0b93c-3ff5-4c64-95b9-4d400b6d5b08 tempest-VolumeQuotaClassesTest-630385983 tempest-VolumeQuotaClassesTest-630385983-project-admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:50.888393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3ac32d9d5642198d60ab01d96c738d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.889491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3ac32d9d5642198d60ab01d96c738d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.892735 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:50.893900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:50.894307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:50.894690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] PATH_INFO: `/v3/users/c7bee406f0e241c7a14adcec2c0600d9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:50.896681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bee406f0e241c7a14adcec2c0600d9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:50.899679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:50.907003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3ac32d9d5642198d60ab01d96c738d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.942909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8060c271-4c71-48e1-aa75-61e938190f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3ac32d9d5642198d60ab01d96c738d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.963764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1272/2550] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:50 2022] DELETE /identity/v3/users/4c3ac32d9d5642198d60ab01d96c738d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:50.965532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba498-e5b6-4e51-8ac5-b5fcca64d80f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:50.965917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba498-e5b6-4e51-8ac5-b5fcca64d80f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:50.966225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bba498-e5b6-4e51-8ac5-b5fcca64d80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:50.990959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bee406f0e241c7a14adcec2c0600d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:50.991573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d5bba498-e5b6-4e51-8ac5-b5fcca64d80f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:50.992895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d5bba498-e5b6-4e51-8ac5-b5fcca64d80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:51.008630 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bee406f0e241c7a14adcec2c0600d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.009375 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bee406f0e241c7a14adcec2c0600d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.030075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bee406f0e241c7a14adcec2c0600d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.040546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-94e4dbff-ce70-4067-952e-e9c2f1d9a03c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bee406f0e241c7a14adcec2c0600d9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.050022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1279/2551] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:50 2022] DELETE /identity/v3/users/c7bee406f0e241c7a14adcec2c0600d9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:51.055769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8dda-0b2d-4873-bb8d-ba71f7b250c0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:51.056075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8dda-0b2d-4873-bb8d-ba71f7b250c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:51.056669 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8dda-0b2d-4873-bb8d-ba71f7b250c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:51.084837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-993d8dda-0b2d-4873-bb8d-ba71f7b250c0 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:51.085074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1273/2552] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:51.086264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-993d8dda-0b2d-4873-bb8d-ba71f7b250c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:51.095956 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bba498-e5b6-4e51-8ac5-b5fcca64d80f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:51.097888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d5bba498-e5b6-4e51-8ac5-b5fcca64d80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:51.184940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1280/2553] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:51.193077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bba498-e5b6-4e51-8ac5-b5fcca64d80f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:51.196184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8dda-0b2d-4873-bb8d-ba71f7b250c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:51.197293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:51.250864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:51.287093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:51.288210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:51.288675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:51.289263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:51.302646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:51.304252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:51.309322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8dda-0b2d-4873-bb8d-ba71f7b250c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:51.317087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1274/2554] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:39:51.319456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb2e7-669c-48e8-ab5d-8db8022e95b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:51.320664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb2e7-669c-48e8-ab5d-8db8022e95b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:51.321113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb2e7-669c-48e8-ab5d-8db8022e95b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:51.321697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadb2e7-669c-48e8-ab5d-8db8022e95b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:51.347766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadb2e7-669c-48e8-ab5d-8db8022e95b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:51.349251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadb2e7-669c-48e8-ab5d-8db8022e95b2 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:51.367793 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1281/2555] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:39:51.841566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:51.843182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:51.857818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f020b-3c58-4edf-a9e8-c1187cbc6559 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:51.864410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:51.865144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:51.865544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:51.865959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] PATH_INFO: `/v3/projects/8fb1ef095da946fb9bce9228c157a74b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:51.873398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb1ef095da946fb9bce9228c157a74b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:51.875787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:51.937316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb1ef095da946fb9bce9228c157a74b', 'request_id': 'req-16eee179-ccf1-46f9-8927-5e1832652ab3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.947592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb1ef095da946fb9bce9228c157a74b', 'request_id': 'req-16eee179-ccf1-46f9-8927-5e1832652ab3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.954599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb1ef095da946fb9bce9228c157a74b', 'request_id': 'req-16eee179-ccf1-46f9-8927-5e1832652ab3'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.971382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] The token cache is being invalidate because project 8fb1ef095da946fb9bce9228c157a74b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:51.971717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.972527 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:51.974229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1275/2556] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:51 2022] DELETE /identity/v3/projects/8fb1ef095da946fb9bce9228c157a74b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:51.978240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb2e7-669c-48e8-ab5d-8db8022e95b2 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:51.991899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadb2e7-669c-48e8-ab5d-8db8022e95b2 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:52.059376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:52.060024 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.060311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.060595 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] PATH_INFO: `/v3/users/4a65fc0220e54175a8c55793dbbd2b0a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.061813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a65fc0220e54175a8c55793dbbd2b0a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:52.063767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:52.116087 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a65fc0220e54175a8c55793dbbd2b0a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.129391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a65fc0220e54175a8c55793dbbd2b0a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.130186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a65fc0220e54175a8c55793dbbd2b0a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.138946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:52.140465 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:52.142798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a65fc0220e54175a8c55793dbbd2b0a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.152306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a65fc0220e54175a8c55793dbbd2b0a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.154272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eee179-ccf1-46f9-8927-5e1832652ab3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:52.159412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1282/2557] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:51 2022] DELETE /identity/v3/users/4a65fc0220e54175a8c55793dbbd2b0a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:52.159721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:52.160537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.160824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.161123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] PATH_INFO: `/v3/projects/f13b3c0bb11e4c6c9f3064165847254c` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.162408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13b3c0bb11e4c6c9f3064165847254c)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:52.164362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:52.216311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13b3c0bb11e4c6c9f3064165847254c', 'request_id': 'req-42412d38-d557-4b7c-a5af-aed87ed39ed2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.223967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13b3c0bb11e4c6c9f3064165847254c', 'request_id': 'req-42412d38-d557-4b7c-a5af-aed87ed39ed2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.230165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13b3c0bb11e4c6c9f3064165847254c', 'request_id': 'req-42412d38-d557-4b7c-a5af-aed87ed39ed2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.243075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] The token cache is being invalidate because project f13b3c0bb11e4c6c9f3064165847254c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:52.243373 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.244041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.245843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1276/2558] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:52 2022] DELETE /identity/v3/projects/f13b3c0bb11e4c6c9f3064165847254c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:52.249753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:52.251186 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:52.313440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:52.333284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ec52b-775b-4540-bbe2-4fae0ddc3a55 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:52.400562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:52.401202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.401522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.401836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] PATH_INFO: `/v3/users/41be4bba01644a7fb22c88ad68c2481f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.403086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41be4bba01644a7fb22c88ad68c2481f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:52.405166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:52.463612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:52.465947 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:52.470416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41be4bba01644a7fb22c88ad68c2481f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.486242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42412d38-d557-4b7c-a5af-aed87ed39ed2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:52.491319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41be4bba01644a7fb22c88ad68c2481f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.492500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41be4bba01644a7fb22c88ad68c2481f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.496974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:52.498803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.499165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.500026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] PATH_INFO: `/v3/projects/878fe08d009a4399877a4124667421a8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.501402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=878fe08d009a4399877a4124667421a8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:52.504723 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:52.505112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41be4bba01644a7fb22c88ad68c2481f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.514342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ff54ddde-8882-4347-a74c-678207dacaf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41be4bba01644a7fb22c88ad68c2481f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.520078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:52.525813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1283/2559] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:52 2022] DELETE /identity/v3/users/41be4bba01644a7fb22c88ad68c2481f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:52.573001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '878fe08d009a4399877a4124667421a8', 'request_id': 'req-853bf66c-fa1a-4eb9-b069-e8176119dc24'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.581849 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '878fe08d009a4399877a4124667421a8', 'request_id': 'req-853bf66c-fa1a-4eb9-b069-e8176119dc24'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.590907 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '878fe08d009a4399877a4124667421a8', 'request_id': 'req-853bf66c-fa1a-4eb9-b069-e8176119dc24'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.607713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] The token cache is being invalidate because project 878fe08d009a4399877a4124667421a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:52.608732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.609960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-853bf66c-fa1a-4eb9-b069-e8176119dc24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:52.617189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1277/2560] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:52 2022] DELETE /identity/v3/projects/878fe08d009a4399877a4124667421a8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:52.620711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0391b-d6b6-41a4-a73e-f07bbbd435b4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.622038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0391b-d6b6-41a4-a73e-f07bbbd435b4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.623209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0391b-d6b6-41a4-a73e-f07bbbd435b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.648676 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0ba0391b-d6b6-41a4-a73e-f07bbbd435b4 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:52.651847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0ba0391b-d6b6-41a4-a73e-f07bbbd435b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:52.754826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1284/2561] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:52.761863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f6d0-4eab-455f-903a-891121c48782 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.762276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f6d0-4eab-455f-903a-891121c48782 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.762669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f6d0-4eab-455f-903a-891121c48782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.786826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-7788f6d0-4eab-455f-903a-891121c48782 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:52.788763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7788f6d0-4eab-455f-903a-891121c48782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:52.806116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1278/2562] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:52.810214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0391b-d6b6-41a4-a73e-f07bbbd435b4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:52.821866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0391b-d6b6-41a4-a73e-f07bbbd435b4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:52.827183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:52.827802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.828085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.828695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.829352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:39:52.829694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:52.831514 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:52.845830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1285/2563] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:39:52.850519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788f6d0-4eab-455f-903a-891121c48782 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:52.860135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788f6d0-4eab-455f-903a-891121c48782 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:52.865618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46693dab-c2bc-47a9-9ff9-c6bf51afe008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:52.866241 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46693dab-c2bc-47a9-9ff9-c6bf51afe008 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.866525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46693dab-c2bc-47a9-9ff9-c6bf51afe008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.866827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46693dab-c2bc-47a9-9ff9-c6bf51afe008 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.867454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46693dab-c2bc-47a9-9ff9-c6bf51afe008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:52.869442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46693dab-c2bc-47a9-9ff9-c6bf51afe008 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:52.897582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:52.899560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:52.902414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1279/2564] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:52 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:39:52.906484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46693dab-c2bc-47a9-9ff9-c6bf51afe008 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:52.915129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:52.920135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd31a4-8237-473c-be48-0e90f1d8b94e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:52.985968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46693dab-c2bc-47a9-9ff9-c6bf51afe008 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:52.991128 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c6804-6d92-4bb2-b6d6-8e72c9a5cb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:52.991747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6804-6d92-4bb2-b6d6-8e72c9a5cb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.992035 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6804-6d92-4bb2-b6d6-8e72c9a5cb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.992304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c6804-6d92-4bb2-b6d6-8e72c9a5cb24 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.992990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c6804-6d92-4bb2-b6d6-8e72c9a5cb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:52.994991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c6804-6d92-4bb2-b6d6-8e72c9a5cb24 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:52.995700 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:52.996407 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:52.996637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:52.996921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] PATH_INFO: `/v3/projects/7104850369994c3c83a5e0dcc71074b7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:52.998141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7104850369994c3c83a5e0dcc71074b7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.000402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.020379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1280/2565] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:52 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:39:53.024649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c6804-6d92-4bb2-b6d6-8e72c9a5cb24 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.036115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c6804-6d92-4bb2-b6d6-8e72c9a5cb24 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.041278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7091-a741-4491-937c-ffc4a3846ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.042009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7091-a741-4491-937c-ffc4a3846ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.042331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7091-a741-4491-937c-ffc4a3846ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.042610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd7091-a741-4491-937c-ffc4a3846ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.043386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd7091-a741-4491-937c-ffc4a3846ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.045297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd7091-a741-4491-937c-ffc4a3846ee2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.056194 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1281/2566] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:53.060136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7091-a741-4491-937c-ffc4a3846ee2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.063445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7104850369994c3c83a5e0dcc71074b7', 'request_id': 'req-e2682c14-f9ed-441c-b636-4c186e31be71'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:53.070628 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7104850369994c3c83a5e0dcc71074b7', 'request_id': 'req-e2682c14-f9ed-441c-b636-4c186e31be71'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:53.071077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd7091-a741-4491-937c-ffc4a3846ee2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.079521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7104850369994c3c83a5e0dcc71074b7', 'request_id': 'req-e2682c14-f9ed-441c-b636-4c186e31be71'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:53.093207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] The token cache is being invalidate because project 7104850369994c3c83a5e0dcc71074b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:39:53.093538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:53.094238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2682c14-f9ed-441c-b636-4c186e31be71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:53.096048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1286/2567] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:39:52 2022] DELETE /identity/v3/projects/7104850369994c3c83a5e0dcc71074b7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:53.101535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e4d0-8f33-4ad6-8d05-7664871841d5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.101830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e4d0-8f33-4ad6-8d05-7664871841d5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.102119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877e4d0-8f33-4ad6-8d05-7664871841d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.122748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7877e4d0-8f33-4ad6-8d05-7664871841d5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:53.123945 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7877e4d0-8f33-4ad6-8d05-7664871841d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:53.132783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:53.149147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.150025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.150307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.150623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.151233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.153029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.166278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1282/2568] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:53.171498 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.173243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:53.186060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaae2c-f908-4e4b-b66d-ae8232e60028 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.191447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f6d8a-4723-4860-abb3-e04ebe82ac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.192123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f6d8a-4723-4860-abb3-e04ebe82ac39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.192399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f6d8a-4723-4860-abb3-e04ebe82ac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.192399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f6d8a-4723-4860-abb3-e04ebe82ac39 admin admin] PATH_INFO: `/v3/projects/ffac9d0074c14d7b84f03396a9edf2fd/users/ac723a0a1f46465b8cde4c2a94637fde/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.203647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1287/2569] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:53.210362 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f6d8a-4723-4860-abb3-e04ebe82ac39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffac9d0074c14d7b84f03396a9edf2fd, user_id=ac723a0a1f46465b8cde4c2a94637fde, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.212105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c84d61-7fbc-4226-a040-57347d6a6199 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.212440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c84d61-7fbc-4226-a040-57347d6a6199 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.212718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c84d61-7fbc-4226-a040-57347d6a6199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.213377 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f6d8a-4723-4860-abb3-e04ebe82ac39 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.227345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1283/2570] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:53 2022] PUT /identity/v3/projects/ffac9d0074c14d7b84f03396a9edf2fd/users/ac723a0a1f46465b8cde4c2a94637fde/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:53.238638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64927f-7f0d-4182-bbf8-f36e63db5d6f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.238951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64927f-7f0d-4182-bbf8-f36e63db5d6f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.239234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64927f-7f0d-4182-bbf8-f36e63db5d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.244465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-59c84d61-7fbc-4226-a040-57347d6a6199 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:53.245743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-59c84d61-7fbc-4226-a040-57347d6a6199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:53.270224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ef64927f-7f0d-4182-bbf8-f36e63db5d6f None None] MFA Rules not processed for user `ac723a0a1f46465b8cde4c2a94637fde`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:53.271571 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ef64927f-7f0d-4182-bbf8-f36e63db5d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:53.282304 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:53.319684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1288/2571] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:53.324102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c84d61-7fbc-4226-a040-57347d6a6199 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.334965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c84d61-7fbc-4226-a040-57347d6a6199 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.340625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.341499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.341786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.342065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.342726 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:39:53.343042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.344687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.354568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1284/2572] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:53.357417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1289/2573] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:39:53.359179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64927f-7f0d-4182-bbf8-f36e63db5d6f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.361562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.371214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64927f-7f0d-4182-bbf8-f36e63db5d6f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.371822 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f108f11-ec56-484c-bb5b-ef9e9236df03 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.376176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9fe84-daa5-4e37-8a1f-08b92685f5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.376671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccb0b4-fbfb-4fd0-9d19-fbde3456be41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.376802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9fe84-daa5-4e37-8a1f-08b92685f5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.377019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9fe84-daa5-4e37-8a1f-08b92685f5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.377893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9fe84-daa5-4e37-8a1f-08b92685f5b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.378021 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb0b4-fbfb-4fd0-9d19-fbde3456be41 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.378230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb0b4-fbfb-4fd0-9d19-fbde3456be41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.378602 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9fe84-daa5-4e37-8a1f-08b92685f5b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.378734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccb0b4-fbfb-4fd0-9d19-fbde3456be41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.379316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccb0b4-fbfb-4fd0-9d19-fbde3456be41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.380599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9fe84-daa5-4e37-8a1f-08b92685f5b9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.381249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccb0b4-fbfb-4fd0-9d19-fbde3456be41 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.411993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1290/2574] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:39:53.415407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1285/2575] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:53.416199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccb0b4-fbfb-4fd0-9d19-fbde3456be41 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.416331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9fe84-daa5-4e37-8a1f-08b92685f5b9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.479562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9fe84-daa5-4e37-8a1f-08b92685f5b9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.479763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccb0b4-fbfb-4fd0-9d19-fbde3456be41 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.484474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67f82e-1b7f-4a99-a60b-2cb145da36e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.485052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545ba7-bec7-4004-976c-37ddbb6d12f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.485198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67f82e-1b7f-4a99-a60b-2cb145da36e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.485382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67f82e-1b7f-4a99-a60b-2cb145da36e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.485683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67f82e-1b7f-4a99-a60b-2cb145da36e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.485801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545ba7-bec7-4004-976c-37ddbb6d12f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.485970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545ba7-bec7-4004-976c-37ddbb6d12f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.486255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545ba7-bec7-4004-976c-37ddbb6d12f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.486367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67f82e-1b7f-4a99-a60b-2cb145da36e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.487959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545ba7-bec7-4004-976c-37ddbb6d12f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.489347 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67f82e-1b7f-4a99-a60b-2cb145da36e1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.489496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545ba7-bec7-4004-976c-37ddbb6d12f4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.516190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1291/2576] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:53.516430 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1286/2577] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:53.522080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545ba7-bec7-4004-976c-37ddbb6d12f4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.522268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67f82e-1b7f-4a99-a60b-2cb145da36e1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.532385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545ba7-bec7-4004-976c-37ddbb6d12f4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.532641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67f82e-1b7f-4a99-a60b-2cb145da36e1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.538084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b33fb-7934-48e7-862c-92e656cbe996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.538736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b33fb-7934-48e7-862c-92e656cbe996 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.539152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b33fb-7934-48e7-862c-92e656cbe996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.539436 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b33fb-7934-48e7-862c-92e656cbe996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.539555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ac8f-2f77-4e63-bd29-c17d25fed1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.540091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b33fb-7934-48e7-862c-92e656cbe996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.540210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ac8f-2f77-4e63-bd29-c17d25fed1fe admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.540427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ac8f-2f77-4e63-bd29-c17d25fed1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.540706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ac8f-2f77-4e63-bd29-c17d25fed1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.541378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3ac8f-2f77-4e63-bd29-c17d25fed1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.541796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b33fb-7934-48e7-862c-92e656cbe996 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.542998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3ac8f-2f77-4e63-bd29-c17d25fed1fe admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.554315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1292/2578] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:53.554572 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1287/2579] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:53.558408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b33fb-7934-48e7-862c-92e656cbe996 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.558535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ac8f-2f77-4e63-bd29-c17d25fed1fe admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.569070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ac8f-2f77-4e63-bd29-c17d25fed1fe admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.569270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b33fb-7934-48e7-862c-92e656cbe996 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.573981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0470e-1b2e-488b-ad55-aecd1f56918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.574132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15851b-fc97-4f2f-b58c-5e859248519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.574674 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15851b-fc97-4f2f-b58c-5e859248519d admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.574809 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0470e-1b2e-488b-ad55-aecd1f56918c admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.574958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15851b-fc97-4f2f-b58c-5e859248519d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.575065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0470e-1b2e-488b-ad55-aecd1f56918c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.575237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15851b-fc97-4f2f-b58c-5e859248519d admin admin] PATH_INFO: `/v3/projects/b558c75a4ff345fb90a9a136fcd3a80a/users/78711d5cb11a4b35bc61cc0c2098353b/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.575345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0470e-1b2e-488b-ad55-aecd1f56918c admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.575880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0470e-1b2e-488b-ad55-aecd1f56918c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.577757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0470e-1b2e-488b-ad55-aecd1f56918c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.580052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:53.591331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15851b-fc97-4f2f-b58c-5e859248519d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b558c75a4ff345fb90a9a136fcd3a80a, user_id=78711d5cb11a4b35bc61cc0c2098353b, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.592361 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1288/2580] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:53.594176 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15851b-fc97-4f2f-b58c-5e859248519d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.597248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0470e-1b2e-488b-ad55-aecd1f56918c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.608182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0470e-1b2e-488b-ad55-aecd1f56918c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.613166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1293/2581] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:53 2022] PUT /identity/v3/projects/b558c75a4ff345fb90a9a136fcd3a80a/users/78711d5cb11a4b35bc61cc0c2098353b/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:53.614052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15851b-fc97-4f2f-b58c-5e859248519d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.673305 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4d335-ccc0-40fe-9f2b-a37cd3773f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.673926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4d335-ccc0-40fe-9f2b-a37cd3773f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.674906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4d335-ccc0-40fe-9f2b-a37cd3773f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.675295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4d335-ccc0-40fe-9f2b-a37cd3773f42 admin admin] PATH_INFO: `/v3/projects/a1a136321c2642448800c23cd9e762c8/users/b25e5a6cdfe640758c4385b5f1b4faec/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.677800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:53.682997 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15851b-fc97-4f2f-b58c-5e859248519d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.690627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726679a-43f0-422c-a1c7-4ba13b27bc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.691340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726679a-43f0-422c-a1c7-4ba13b27bc83 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.691580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726679a-43f0-422c-a1c7-4ba13b27bc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.691841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726679a-43f0-422c-a1c7-4ba13b27bc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.692498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726679a-43f0-422c-a1c7-4ba13b27bc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.693225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4d335-ccc0-40fe-9f2b-a37cd3773f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a136321c2642448800c23cd9e762c8, user_id=b25e5a6cdfe640758c4385b5f1b4faec, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.695369 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726679a-43f0-422c-a1c7-4ba13b27bc83 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.696221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f4d335-ccc0-40fe-9f2b-a37cd3773f42 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.713654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1289/2582] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:53 2022] PUT /identity/v3/projects/a1a136321c2642448800c23cd9e762c8/users/b25e5a6cdfe640758c4385b5f1b4faec/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:53.713900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1294/2583] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:53.718387 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4d335-ccc0-40fe-9f2b-a37cd3773f42 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.723344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ba19b-0c97-46ac-a2aa-67a03b161fac None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.723642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ba19b-0c97-46ac-a2aa-67a03b161fac None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.723926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ba19b-0c97-46ac-a2aa-67a03b161fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.760062 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a58ba19b-0c97-46ac-a2aa-67a03b161fac None None] MFA Rules not processed for user `b25e5a6cdfe640758c4385b5f1b4faec`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:53.761517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a58ba19b-0c97-46ac-a2aa-67a03b161fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:53.794967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4d335-ccc0-40fe-9f2b-a37cd3773f42 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.802047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6555360-0811-4203-9e1c-1e393c99aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.803163 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555360-0811-4203-9e1c-1e393c99aaf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.803657 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555360-0811-4203-9e1c-1e393c99aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.804157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555360-0811-4203-9e1c-1e393c99aaf3 admin admin] PATH_INFO: `/v3/projects/b558c75a4ff345fb90a9a136fcd3a80a/users/78711d5cb11a4b35bc61cc0c2098353b/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.807643 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6555360-0811-4203-9e1c-1e393c99aaf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b558c75a4ff345fb90a9a136fcd3a80a, user_id=78711d5cb11a4b35bc61cc0c2098353b, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.813203 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6555360-0811-4203-9e1c-1e393c99aaf3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.832975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1290/2584] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:53 2022] PUT /identity/v3/projects/b558c75a4ff345fb90a9a136fcd3a80a/users/78711d5cb11a4b35bc61cc0c2098353b/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:53.844598 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eda7e-8197-4ff8-870c-414dd6fefcc5 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.844885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eda7e-8197-4ff8-870c-414dd6fefcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.845164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eda7e-8197-4ff8-870c-414dd6fefcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.848227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1295/2585] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:53.852527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ba19b-0c97-46ac-a2aa-67a03b161fac None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:53.870952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ba19b-0c97-46ac-a2aa-67a03b161fac None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:53.880442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e54eda7e-8197-4ff8-870c-414dd6fefcc5 None None] MFA Rules not processed for user `78711d5cb11a4b35bc61cc0c2098353b`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:53.882521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e54eda7e-8197-4ff8-870c-414dd6fefcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:53.959821 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89827cf-efe6-4dbf-832a-664b8f6e4581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:53.960457 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89827cf-efe6-4dbf-832a-664b8f6e4581 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:53.960757 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89827cf-efe6-4dbf-832a-664b8f6e4581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:53.962065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89827cf-efe6-4dbf-832a-664b8f6e4581 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:53.962065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89827cf-efe6-4dbf-832a-664b8f6e4581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:53.963715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89827cf-efe6-4dbf-832a-664b8f6e4581 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:53.980319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:53.991243 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1291/2586] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:54.002242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1296/2587] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:39:53 2022] POST /identity/v3/projects => generated 399 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:39:54.004006 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54eda7e-8197-4ff8-870c-414dd6fefcc5 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.004139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89827cf-efe6-4dbf-832a-664b8f6e4581 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.005419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e54eda7e-8197-4ff8-870c-414dd6fefcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:54.099755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54eda7e-8197-4ff8-870c-414dd6fefcc5 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.109966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89827cf-efe6-4dbf-832a-664b8f6e4581 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.115810 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c795282-7abf-4c87-8110-c23a6c7c7815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.116440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c795282-7abf-4c87-8110-c23a6c7c7815 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.116723 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c795282-7abf-4c87-8110-c23a6c7c7815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.117006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c795282-7abf-4c87-8110-c23a6c7c7815 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.117683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c795282-7abf-4c87-8110-c23a6c7c7815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.119625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c795282-7abf-4c87-8110-c23a6c7c7815 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.132129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:54.145928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1297/2588] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:39:54 2022] POST /identity/v3/users => generated 306 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:39:54.149800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c795282-7abf-4c87-8110-c23a6c7c7815 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.159518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420c20d-c178-479d-a299-de3ff983aca3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.160150 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420c20d-c178-479d-a299-de3ff983aca3 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.160440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420c20d-c178-479d-a299-de3ff983aca3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.160715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420c20d-c178-479d-a299-de3ff983aca3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.160833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c795282-7abf-4c87-8110-c23a6c7c7815 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.165906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660aacd-2835-4f63-88a2-d2c0969163cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.166548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660aacd-2835-4f63-88a2-d2c0969163cf admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.166840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660aacd-2835-4f63-88a2-d2c0969163cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.167116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660aacd-2835-4f63-88a2-d2c0969163cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.167798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660aacd-2835-4f63-88a2-d2c0969163cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.169403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660aacd-2835-4f63-88a2-d2c0969163cf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.182123 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1298/2589] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:39:54.186213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660aacd-2835-4f63-88a2-d2c0969163cf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.197355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660aacd-2835-4f63-88a2-d2c0969163cf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.205043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fe0eb-7be3-41c1-9758-ca6d6f1f4280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.206751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fe0eb-7be3-41c1-9758-ca6d6f1f4280 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.207033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fe0eb-7be3-41c1-9758-ca6d6f1f4280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.207420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fe0eb-7be3-41c1-9758-ca6d6f1f4280 admin admin] PATH_INFO: `/v3/projects/57f47303ded74b6db099a9bde8fe048b/users/a6bbb622765345919be2585a1eccb3bd/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.227169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fe0eb-7be3-41c1-9758-ca6d6f1f4280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f47303ded74b6db099a9bde8fe048b, user_id=a6bbb622765345919be2585a1eccb3bd, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.230107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800fe0eb-7be3-41c1-9758-ca6d6f1f4280 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.230254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420c20d-c178-479d-a299-de3ff983aca3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.231586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420c20d-c178-479d-a299-de3ff983aca3 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.242995 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1292/2590] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:54.251373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1299/2591] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:54 2022] PUT /identity/v3/projects/57f47303ded74b6db099a9bde8fe048b/users/a6bbb622765345919be2585a1eccb3bd/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:54.255906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420c20d-c178-479d-a299-de3ff983aca3 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.267193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420c20d-c178-479d-a299-de3ff983aca3 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.345747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebe9d1-bae1-4662-b332-b038a085d47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.345747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebe9d1-bae1-4662-b332-b038a085d47f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.345747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebe9d1-bae1-4662-b332-b038a085d47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.345747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebe9d1-bae1-4662-b332-b038a085d47f admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.346768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebe9d1-bae1-4662-b332-b038a085d47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.346768 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebe9d1-bae1-4662-b332-b038a085d47f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.353958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:54.355231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fe0eb-7be3-41c1-9758-ca6d6f1f4280 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.361633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1293/2592] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:39:54.366761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebe9d1-bae1-4662-b332-b038a085d47f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.382267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebe9d1-bae1-4662-b332-b038a085d47f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.387641 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3a069-3271-4d7b-a150-749a03d94b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.388454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3a069-3271-4d7b-a150-749a03d94b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.388918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3a069-3271-4d7b-a150-749a03d94b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.389200 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3a069-3271-4d7b-a150-749a03d94b35 admin admin] PATH_INFO: `/v3/projects/57f47303ded74b6db099a9bde8fe048b/users/a6bbb622765345919be2585a1eccb3bd/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.391276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3a069-3271-4d7b-a150-749a03d94b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f47303ded74b6db099a9bde8fe048b, user_id=a6bbb622765345919be2585a1eccb3bd, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.394065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3a069-3271-4d7b-a150-749a03d94b35 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.407255 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1294/2593] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:39:54 2022] PUT /identity/v3/projects/57f47303ded74b6db099a9bde8fe048b/users/a6bbb622765345919be2585a1eccb3bd/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:39:54.420822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da720c15-8811-4af6-a55b-00599a244dae None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.421195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da720c15-8811-4af6-a55b-00599a244dae None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.421618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da720c15-8811-4af6-a55b-00599a244dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.435488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fe0eb-7be3-41c1-9758-ca6d6f1f4280 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.452351 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-da720c15-8811-4af6-a55b-00599a244dae None None] MFA Rules not processed for user `a6bbb622765345919be2585a1eccb3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:54.453647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-da720c15-8811-4af6-a55b-00599a244dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:54.489088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63be718-769d-4ada-b083-4e7ef791ece2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.489771 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be718-769d-4ada-b083-4e7ef791ece2 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.490055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be718-769d-4ada-b083-4e7ef791ece2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.490331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be718-769d-4ada-b083-4e7ef791ece2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.532131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1295/2594] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:54 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:39:54.543957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da720c15-8811-4af6-a55b-00599a244dae None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.545471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-da720c15-8811-4af6-a55b-00599a244dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:54.557484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63be718-769d-4ada-b083-4e7ef791ece2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.558633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da720c15-8811-4af6-a55b-00599a244dae None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.558983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63be718-769d-4ada-b083-4e7ef791ece2 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.562617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d29d91-e72e-43d7-b62a-047307eef15c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.563234 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d29d91-e72e-43d7-b62a-047307eef15c service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.563521 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d29d91-e72e-43d7-b62a-047307eef15c service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.563796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d29d91-e72e-43d7-b62a-047307eef15c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.568409 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1300/2595] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:54.573298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d29d91-e72e-43d7-b62a-047307eef15c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.574712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d29d91-e72e-43d7-b62a-047307eef15c service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.584843 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1296/2596] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:54 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:39:54.665052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63be718-769d-4ada-b083-4e7ef791ece2 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:54.673755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63be718-769d-4ada-b083-4e7ef791ece2 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:54.677784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e079e-2c72-489f-8bd2-fdc876e2d0e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:54.678391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e079e-2c72-489f-8bd2-fdc876e2d0e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:54.678668 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e079e-2c72-489f-8bd2-fdc876e2d0e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:54.678946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e079e-2c72-489f-8bd2-fdc876e2d0e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:54.710257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:54.743066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e079e-2c72-489f-8bd2-fdc876e2d0e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:54.744531 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e079e-2c72-489f-8bd2-fdc876e2d0e0 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:54.754320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1301/2597] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:54 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:59.203303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d29d91-e72e-43d7-b62a-047307eef15c service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:59.290327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d29d91-e72e-43d7-b62a-047307eef15c service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:59.298583 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:59.299483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:59.300000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:59.300396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] PATH_INFO: `/v3/users/b25e5a6cdfe640758c4385b5f1b4faec` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:59.302591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b25e5a6cdfe640758c4385b5f1b4faec)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:59.304922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:59.382103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b25e5a6cdfe640758c4385b5f1b4faec'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:59.383784 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b25e5a6cdfe640758c4385b5f1b4faec'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:59.384534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b25e5a6cdfe640758c4385b5f1b4faec'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:59.394315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b25e5a6cdfe640758c4385b5f1b4faec'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:59.402592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:39:59.420842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b25e5a6cdfe640758c4385b5f1b4faec'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:39:59.446832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1297/2598] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:59 2022] DELETE /identity/v3/users/b25e5a6cdfe640758c4385b5f1b4faec => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:39:59.451500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9830ae-b523-4998-8943-e216a1a5fe6f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:59.452013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9830ae-b523-4998-8943-e216a1a5fe6f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:59.452145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9830ae-b523-4998-8943-e216a1a5fe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:59.473787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ca9830ae-b523-4998-8943-e216a1a5fe6f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:39:59.475067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ca9830ae-b523-4998-8943-e216a1a5fe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:59.550264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1302/2599] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:39:59.562020 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:59.564332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:59.669917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92b548-f1b2-4bfa-b007-3102f753af5c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:59.678516 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27abb9-029a-4d6e-9001-87a0c7016816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:59.680293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27abb9-029a-4d6e-9001-87a0c7016816 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:59.680293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27abb9-029a-4d6e-9001-87a0c7016816 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:59.680898 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27abb9-029a-4d6e-9001-87a0c7016816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:59.699288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27abb9-029a-4d6e-9001-87a0c7016816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:59.702333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27abb9-029a-4d6e-9001-87a0c7016816 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:59.725978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1298/2600] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:39:59.840414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9830ae-b523-4998-8943-e216a1a5fe6f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:39:59.844561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ca9830ae-b523-4998-8943-e216a1a5fe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:39:59.872342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9830ae-b523-4998-8943-e216a1a5fe6f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:39:59.882376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:39:59.883564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:39:59.884364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:39:59.884929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] PATH_INFO: `/v3/users/ac723a0a1f46465b8cde4c2a94637fde` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:39:59.887270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac723a0a1f46465b8cde4c2a94637fde)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:39:59.891406 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:39:59.987359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac723a0a1f46465b8cde4c2a94637fde'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.012524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac723a0a1f46465b8cde4c2a94637fde'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.013951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac723a0a1f46465b8cde4c2a94637fde'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.028816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac723a0a1f46465b8cde4c2a94637fde'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.041906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac723a0a1f46465b8cde4c2a94637fde'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.059341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1303/2601] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:39:59 2022] DELETE /identity/v3/users/ac723a0a1f46465b8cde4c2a94637fde => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:00.062147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f6a5c-ed40-4cce-9ecb-ba294abf2621 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:00.062957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f6a5c-ed40-4cce-9ecb-ba294abf2621 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:00.063550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f6a5c-ed40-4cce-9ecb-ba294abf2621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:00.105794 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-6f3f6a5c-ed40-4cce-9ecb-ba294abf2621 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:00.109654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6f3f6a5c-ed40-4cce-9ecb-ba294abf2621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:00.155885 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:00.157991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:00.270801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:00.271590 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1299/2602] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:00.287474 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f6a5c-ed40-4cce-9ecb-ba294abf2621 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:00.290199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6f3f6a5c-ed40-4cce-9ecb-ba294abf2621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:00.303859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc6e55-fffd-4b53-96cb-ddd1e85f673f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:00.309604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:00.310312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:00.311258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:00.311631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] PATH_INFO: `/v3/projects/a1a136321c2642448800c23cd9e762c8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:00.313038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a136321c2642448800c23cd9e762c8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:00.315640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:00.372814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f6a5c-ed40-4cce-9ecb-ba294abf2621 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:00.390384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a136321c2642448800c23cd9e762c8', 'request_id': 'req-46522f0a-85a7-4c19-b309-b8204be1a3ae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.398579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a136321c2642448800c23cd9e762c8', 'request_id': 'req-46522f0a-85a7-4c19-b309-b8204be1a3ae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.412741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a136321c2642448800c23cd9e762c8', 'request_id': 'req-46522f0a-85a7-4c19-b309-b8204be1a3ae'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.426626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:00.433120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] The token cache is being invalidate because project a1a136321c2642448800c23cd9e762c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:00.433604 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.434615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.437425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1304/2603] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:00 2022] DELETE /identity/v3/projects/a1a136321c2642448800c23cd9e762c8 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:00.441359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:00.443183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:00.450335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:00.451116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:00.451464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:00.451743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:00.454689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:00.537559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:00.539052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:00.543033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46522f0a-85a7-4c19-b309-b8204be1a3ae admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:00.548228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:00.548943 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:00.550435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1300/2604] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:00.550605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:00.550605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] PATH_INFO: `/v3/users/a6bbb622765345919be2585a1eccb3bd` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:00.551722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6bbb622765345919be2585a1eccb3bd)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:00.553722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:00.613998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6bbb622765345919be2585a1eccb3bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.633906 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6bbb622765345919be2585a1eccb3bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.634824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6bbb622765345919be2585a1eccb3bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.645122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6bbb622765345919be2585a1eccb3bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.647806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:00.656820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6bbb622765345919be2585a1eccb3bd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:00.671603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1305/2605] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:00 2022] DELETE /identity/v3/users/a6bbb622765345919be2585a1eccb3bd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:00.948880 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:00.951811 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:00.974477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5461da-e930-42f7-880d-c8e334acbedd service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:01.062670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:01.063333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:01.063613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:01.063895 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] PATH_INFO: `/v3/projects/ffac9d0074c14d7b84f03396a9edf2fd` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:01.066259 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffac9d0074c14d7b84f03396a9edf2fd)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:01.068239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:01.100332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:01.102489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:01.129096 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0de81-d269-4860-8121-321586bcf9c2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:01.140378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffac9d0074c14d7b84f03396a9edf2fd', 'request_id': 'req-34626e7d-c82f-42b1-bc8f-62b33e64bab7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.148395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffac9d0074c14d7b84f03396a9edf2fd', 'request_id': 'req-34626e7d-c82f-42b1-bc8f-62b33e64bab7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.160976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffac9d0074c14d7b84f03396a9edf2fd', 'request_id': 'req-34626e7d-c82f-42b1-bc8f-62b33e64bab7'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.189291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] The token cache is being invalidate because project ffac9d0074c14d7b84f03396a9edf2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:01.190129 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.190859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.192560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1301/2606] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:00 2022] DELETE /identity/v3/projects/ffac9d0074c14d7b84f03396a9edf2fd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:01.197331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:01.199672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:01.225498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:01.226615 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:01.226961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:01.227316 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] PATH_INFO: `/v3/projects/57f47303ded74b6db099a9bde8fe048b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:01.228715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f47303ded74b6db099a9bde8fe048b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:01.230795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:01.232338 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:01.301901 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34626e7d-c82f-42b1-bc8f-62b33e64bab7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:01.323517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f47303ded74b6db099a9bde8fe048b', 'request_id': 'req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.334363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f47303ded74b6db099a9bde8fe048b', 'request_id': 'req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.355174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f47303ded74b6db099a9bde8fe048b', 'request_id': 'req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.397771 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:01.398758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:01.399047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:01.399451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] PATH_INFO: `/v3/users/78711d5cb11a4b35bc61cc0c2098353b` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:01.401493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78711d5cb11a4b35bc61cc0c2098353b)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:01.404011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:01.412106 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] The token cache is being invalidate because project 57f47303ded74b6db099a9bde8fe048b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:01.413396 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.416274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b0c96f87-d74d-4fc3-a0b8-0d756e4a046d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.432097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1306/2607] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:01 2022] DELETE /identity/v3/projects/57f47303ded74b6db099a9bde8fe048b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:01.437237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f8afa-3807-40c7-8483-7ed0490a702a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:01.437555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f8afa-3807-40c7-8483-7ed0490a702a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:01.437839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f8afa-3807-40c7-8483-7ed0490a702a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:01.488511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-696f8afa-3807-40c7-8483-7ed0490a702a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:01.491709 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-696f8afa-3807-40c7-8483-7ed0490a702a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:01.519092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1307/2608] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:01.526974 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4649368-4879-4a23-aa40-354b95ddfae3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:01.528718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4649368-4879-4a23-aa40-354b95ddfae3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:01.529286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4649368-4879-4a23-aa40-354b95ddfae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:01.538036 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78711d5cb11a4b35bc61cc0c2098353b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.542968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78711d5cb11a4b35bc61cc0c2098353b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.543955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78711d5cb11a4b35bc61cc0c2098353b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.562705 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78711d5cb11a4b35bc61cc0c2098353b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.572882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f4649368-4879-4a23-aa40-354b95ddfae3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:01.575203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f4649368-4879-4a23-aa40-354b95ddfae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:01.585303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78711d5cb11a4b35bc61cc0c2098353b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:01.587788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1308/2609] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:01.594995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4649368-4879-4a23-aa40-354b95ddfae3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:01.616171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4649368-4879-4a23-aa40-354b95ddfae3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:01.622954 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1302/2610] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:01 2022] DELETE /identity/v3/users/78711d5cb11a4b35bc61cc0c2098353b => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:01.737039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:01.737927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:01.738416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:01.739135 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:01.740591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:40:01.741136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:01.743586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:01.765347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1309/2611] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:40:01.771312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:01.797791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8b3f1-78b5-49cf-8520-6867ee83b5b9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:01.807086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05628-1c3f-4f8e-aca1-0714ad84e208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:01.807888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05628-1c3f-4f8e-aca1-0714ad84e208 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:01.808356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05628-1c3f-4f8e-aca1-0714ad84e208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:01.809171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba05628-1c3f-4f8e-aca1-0714ad84e208 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:01.810286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba05628-1c3f-4f8e-aca1-0714ad84e208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:01.813605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba05628-1c3f-4f8e-aca1-0714ad84e208 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:01.867962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1303/2612] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:01 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:01.873707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:01.881824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:01.888758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6060ad-feb9-479c-a6fc-e419a9bc8247 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:01.978580 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:01.979230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:01.979512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:01.979789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:01.980456 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:01.982553 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.017237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1310/2613] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:01 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:02.021121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05628-1c3f-4f8e-aca1-0714ad84e208 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.037361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.040307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:02.041827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba05628-1c3f-4f8e-aca1-0714ad84e208 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.048205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53509eb-7458-426c-98ff-bcd547f4b6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.049565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53509eb-7458-426c-98ff-bcd547f4b6ea admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.049565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53509eb-7458-426c-98ff-bcd547f4b6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.049945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53509eb-7458-426c-98ff-bcd547f4b6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.050438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53509eb-7458-426c-98ff-bcd547f4b6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.053332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53509eb-7458-426c-98ff-bcd547f4b6ea admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.056537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f388b6-7520-4fee-a2a0-e330a5348676 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.072103 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.072789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.073082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.073366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] PATH_INFO: `/v3/projects/b558c75a4ff345fb90a9a136fcd3a80a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.079336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b558c75a4ff345fb90a9a136fcd3a80a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.081491 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.086325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1304/2614] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:02.091932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53509eb-7458-426c-98ff-bcd547f4b6ea admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.104516 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53509eb-7458-426c-98ff-bcd547f4b6ea admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.110169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8014f-9dca-4271-b072-955cee8dd228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.110807 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8014f-9dca-4271-b072-955cee8dd228 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.111101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8014f-9dca-4271-b072-955cee8dd228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.111734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8014f-9dca-4271-b072-955cee8dd228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.113278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8014f-9dca-4271-b072-955cee8dd228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.115307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8014f-9dca-4271-b072-955cee8dd228 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.121638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:02.130238 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1305/2615] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:02.134801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8014f-9dca-4271-b072-955cee8dd228 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.150300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8014f-9dca-4271-b072-955cee8dd228 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.165198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b558c75a4ff345fb90a9a136fcd3a80a', 'request_id': 'req-76e59f62-e850-4790-b91a-99193a035f71'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:02.175361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b558c75a4ff345fb90a9a136fcd3a80a', 'request_id': 'req-76e59f62-e850-4790-b91a-99193a035f71'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:02.185267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b558c75a4ff345fb90a9a136fcd3a80a', 'request_id': 'req-76e59f62-e850-4790-b91a-99193a035f71'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:02.201032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] The token cache is being invalidate because project b558c75a4ff345fb90a9a136fcd3a80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:02.201501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:02.202393 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-76e59f62-e850-4790-b91a-99193a035f71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:02.204229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1311/2616] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:02 2022] DELETE /identity/v3/projects/b558c75a4ff345fb90a9a136fcd3a80a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:02.209589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120ece0-8410-4265-9990-95d7e0e01bea None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.209961 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120ece0-8410-4265-9990-95d7e0e01bea None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.210318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120ece0-8410-4265-9990-95d7e0e01bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.230605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a4417-8985-4c0d-b9c7-0aac3ac6fb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.231295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a4417-8985-4c0d-b9c7-0aac3ac6fb97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.231579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a4417-8985-4c0d-b9c7-0aac3ac6fb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.231878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a4417-8985-4c0d-b9c7-0aac3ac6fb97 admin admin] PATH_INFO: `/v3/projects/1745da99c8b74b04a04fbc70e603b24e/users/4c89afe07c794464afc8606ec2a3bcfe/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.235836 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9120ece0-8410-4265-9990-95d7e0e01bea None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:02.237747 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9120ece0-8410-4265-9990-95d7e0e01bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:02.246033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1312/2617] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:02.248268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a4417-8985-4c0d-b9c7-0aac3ac6fb97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1745da99c8b74b04a04fbc70e603b24e, user_id=4c89afe07c794464afc8606ec2a3bcfe, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.251272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a4417-8985-4c0d-b9c7-0aac3ac6fb97 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.255368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaef0b-ae51-4b2f-bafc-0f7cb166483e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.255837 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaef0b-ae51-4b2f-bafc-0f7cb166483e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.256299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaef0b-ae51-4b2f-bafc-0f7cb166483e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.268560 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1306/2618] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:02 2022] PUT /identity/v3/projects/1745da99c8b74b04a04fbc70e603b24e/users/4c89afe07c794464afc8606ec2a3bcfe/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:02.280185 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a9950-9af4-4f85-ac48-c40207748543 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.280623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a9950-9af4-4f85-ac48-c40207748543 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.281015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a9950-9af4-4f85-ac48-c40207748543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.281361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-32aaef0b-ae51-4b2f-bafc-0f7cb166483e None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:02.282896 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-32aaef0b-ae51-4b2f-bafc-0f7cb166483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:02.319456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-974a9950-9af4-4f85-ac48-c40207748543 None None] MFA Rules not processed for user `4c89afe07c794464afc8606ec2a3bcfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:02.321168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-974a9950-9af4-4f85-ac48-c40207748543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:02.364844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1313/2619] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:02.367866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaef0b-ae51-4b2f-bafc-0f7cb166483e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.382384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaef0b-ae51-4b2f-bafc-0f7cb166483e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.388273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.389098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.389277 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.389507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.390164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:40:02.390494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.392139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.395340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:02.402481 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1307/2620] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:02.407828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a9950-9af4-4f85-ac48-c40207748543 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.409645 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-974a9950-9af4-4f85-ac48-c40207748543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:02.414532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1314/2621] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:40:02.415731 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.424675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a9950-9af4-4f85-ac48-c40207748543 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.428195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649627c-c9ec-44cf-9f90-9b20bebb0a95 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.430119 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cba11-291b-43bd-bcb3-1712b232b368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.430751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cba11-291b-43bd-bcb3-1712b232b368 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.431031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cba11-291b-43bd-bcb3-1712b232b368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.431306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cba11-291b-43bd-bcb3-1712b232b368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.431948 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cba11-291b-43bd-bcb3-1712b232b368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.434410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ccbcf-2e0e-4ec7-9aa0-6d8a4f234d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.434916 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cba11-291b-43bd-bcb3-1712b232b368 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.435119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ccbcf-2e0e-4ec7-9aa0-6d8a4f234d62 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.435415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ccbcf-2e0e-4ec7-9aa0-6d8a4f234d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.435899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ccbcf-2e0e-4ec7-9aa0-6d8a4f234d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.436557 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ccbcf-2e0e-4ec7-9aa0-6d8a4f234d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.438537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ccbcf-2e0e-4ec7-9aa0-6d8a4f234d62 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.477220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1315/2623] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:02.477514 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1308/2623] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:02.482632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cba11-291b-43bd-bcb3-1712b232b368 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.496134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ccbcf-2e0e-4ec7-9aa0-6d8a4f234d62 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.560987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:02.580299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cba11-291b-43bd-bcb3-1712b232b368 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.586100 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5c2b-f6f8-41c3-8699-28d059d05eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.586806 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5c2b-f6f8-41c3-8699-28d059d05eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.587113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5c2b-f6f8-41c3-8699-28d059d05eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.587391 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5c2b-f6f8-41c3-8699-28d059d05eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.588071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d5c2b-f6f8-41c3-8699-28d059d05eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.590132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d5c2b-f6f8-41c3-8699-28d059d05eb8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.612951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ccbcf-2e0e-4ec7-9aa0-6d8a4f234d62 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.615978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1309/2624] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:02.619722 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8358031-c59c-437b-aa3f-24ff990fecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.619980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5c2b-f6f8-41c3-8699-28d059d05eb8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.620652 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8358031-c59c-437b-aa3f-24ff990fecf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.621046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8358031-c59c-437b-aa3f-24ff990fecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.621388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8358031-c59c-437b-aa3f-24ff990fecf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.622171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8358031-c59c-437b-aa3f-24ff990fecf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.626169 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8358031-c59c-437b-aa3f-24ff990fecf8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.632500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5c2b-f6f8-41c3-8699-28d059d05eb8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.638531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330e4c-5a3c-4b73-88bd-ddc2b1b0b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.639603 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330e4c-5a3c-4b73-88bd-ddc2b1b0b82e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.640114 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330e4c-5a3c-4b73-88bd-ddc2b1b0b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.640633 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330e4c-5a3c-4b73-88bd-ddc2b1b0b82e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.641914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96330e4c-5a3c-4b73-88bd-ddc2b1b0b82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.645260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96330e4c-5a3c-4b73-88bd-ddc2b1b0b82e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.657311 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1316/2625] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:02.664701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8358031-c59c-437b-aa3f-24ff990fecf8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.672043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1310/2626] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:02.677120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330e4c-5a3c-4b73-88bd-ddc2b1b0b82e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.684662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8358031-c59c-437b-aa3f-24ff990fecf8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.690762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330e4c-5a3c-4b73-88bd-ddc2b1b0b82e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.692919 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8a90a-b071-47bb-b63d-74d9f331f845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.693591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8a90a-b071-47bb-b63d-74d9f331f845 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.693882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8a90a-b071-47bb-b63d-74d9f331f845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.694227 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8a90a-b071-47bb-b63d-74d9f331f845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.694905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8a90a-b071-47bb-b63d-74d9f331f845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.696561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c8a90a-b071-47bb-b63d-74d9f331f845 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.710120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c23067-08d6-4f28-942a-55f3a74b40f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.710798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23067-08d6-4f28-942a-55f3a74b40f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.711083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23067-08d6-4f28-942a-55f3a74b40f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.711599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23067-08d6-4f28-942a-55f3a74b40f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.712392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c23067-08d6-4f28-942a-55f3a74b40f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.714696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c23067-08d6-4f28-942a-55f3a74b40f2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.723574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1317/2627] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:02.734833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8a90a-b071-47bb-b63d-74d9f331f845 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.735121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1311/2628] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:02.736938 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c23067-08d6-4f28-942a-55f3a74b40f2 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.751695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8a90a-b071-47bb-b63d-74d9f331f845 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.755393 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c23067-08d6-4f28-942a-55f3a74b40f2 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.762032 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467dbb9-1e81-4c58-ab84-a2f3ca21daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.763370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467dbb9-1e81-4c58-ab84-a2f3ca21daab admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.763869 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee336ac-c876-4a88-860d-ca652f1497d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.763992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467dbb9-1e81-4c58-ab84-a2f3ca21daab admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.764561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467dbb9-1e81-4c58-ab84-a2f3ca21daab admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.765037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee336ac-c876-4a88-860d-ca652f1497d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.766764 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee336ac-c876-4a88-860d-ca652f1497d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.766916 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467dbb9-1e81-4c58-ab84-a2f3ca21daab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.767001 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee336ac-c876-4a88-860d-ca652f1497d0 admin admin] PATH_INFO: `/v3/projects/f965eb6d51f143c6a099deff7814ca32/users/c494c20c755246dc9c4b11ae85c5e13d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.770164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467dbb9-1e81-4c58-ab84-a2f3ca21daab admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.790317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee336ac-c876-4a88-860d-ca652f1497d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f965eb6d51f143c6a099deff7814ca32, user_id=c494c20c755246dc9c4b11ae85c5e13d, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.793034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee336ac-c876-4a88-860d-ca652f1497d0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.798160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1318/2629] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:02.801265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467dbb9-1e81-4c58-ab84-a2f3ca21daab admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:02.819344 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467dbb9-1e81-4c58-ab84-a2f3ca21daab admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:02.837419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1312/2630] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:02 2022] PUT /identity/v3/projects/f965eb6d51f143c6a099deff7814ca32/users/c494c20c755246dc9c4b11ae85c5e13d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:02.849277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8a99c-d40c-410b-b5b7-06b7e31b0cf7 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.849277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8a99c-d40c-410b-b5b7-06b7e31b0cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.849763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8a99c-d40c-410b-b5b7-06b7e31b0cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.896523 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-81a8a99c-d40c-410b-b5b7-06b7e31b0cf7 None None] MFA Rules not processed for user `c494c20c755246dc9c4b11ae85c5e13d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:02.898261 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-81a8a99c-d40c-410b-b5b7-06b7e31b0cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:02.922183 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a817e36-1cd1-4a51-a5c0-30f14be77911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:02.922867 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817e36-1cd1-4a51-a5c0-30f14be77911 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.923155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817e36-1cd1-4a51-a5c0-30f14be77911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.923441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817e36-1cd1-4a51-a5c0-30f14be77911 admin admin] PATH_INFO: `/v3/projects/d2e1596319e74fbe909e1be08c59b683/users/4bba3c927c18493eb558e1823ffa78ce/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.942339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a817e36-1cd1-4a51-a5c0-30f14be77911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e1596319e74fbe909e1be08c59b683, user_id=4bba3c927c18493eb558e1823ffa78ce, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:02.945422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a817e36-1cd1-4a51-a5c0-30f14be77911 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:02.966853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1319/2631] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:02 2022] PUT /identity/v3/projects/d2e1596319e74fbe909e1be08c59b683/users/4bba3c927c18493eb558e1823ffa78ce/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:02.977519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fa094-4627-4f6b-81d7-e2d02b70e170 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:02.977796 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fa094-4627-4f6b-81d7-e2d02b70e170 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:02.978086 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fa094-4627-4f6b-81d7-e2d02b70e170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:02.991272 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1313/2632] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:02.995782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8a99c-d40c-410b-b5b7-06b7e31b0cf7 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.010115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ed5fa094-4627-4f6b-81d7-e2d02b70e170 None None] MFA Rules not processed for user `4bba3c927c18493eb558e1823ffa78ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:03.011507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ed5fa094-4627-4f6b-81d7-e2d02b70e170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:03.013005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8a99c-d40c-410b-b5b7-06b7e31b0cf7 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.026658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:03.093018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ca99-9404-489b-bfd2-a7dbb57e7555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.093755 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ca99-9404-489b-bfd2-a7dbb57e7555 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.094041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ca99-9404-489b-bfd2-a7dbb57e7555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.094312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ca99-9404-489b-bfd2-a7dbb57e7555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.095318 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ca99-9404-489b-bfd2-a7dbb57e7555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.095585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1320/2633] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:03.097675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ca99-9404-489b-bfd2-a7dbb57e7555 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.101132 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fa094-4627-4f6b-81d7-e2d02b70e170 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.116354 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fa094-4627-4f6b-81d7-e2d02b70e170 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.122936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c258f-449f-4b43-8978-a2137cee53f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.123170 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c258f-449f-4b43-8978-a2137cee53f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.123427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c258f-449f-4b43-8978-a2137cee53f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.123716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c258f-449f-4b43-8978-a2137cee53f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.124624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c258f-449f-4b43-8978-a2137cee53f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.126606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c258f-449f-4b43-8978-a2137cee53f3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.132277 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1314/2634] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:02 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:03.136278 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ca99-9404-489b-bfd2-a7dbb57e7555 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.150703 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:03.162201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1321/2635] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:03 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:03.168550 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c258f-449f-4b43-8978-a2137cee53f3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.220626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ca99-9404-489b-bfd2-a7dbb57e7555 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.242072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c258f-449f-4b43-8978-a2137cee53f3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.247519 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95291af4-e4a1-4e34-bbf7-c1eef37a231c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.249701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95291af4-e4a1-4e34-bbf7-c1eef37a231c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.249701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95291af4-e4a1-4e34-bbf7-c1eef37a231c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.249701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95291af4-e4a1-4e34-bbf7-c1eef37a231c admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.250324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95291af4-e4a1-4e34-bbf7-c1eef37a231c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.251828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95291af4-e4a1-4e34-bbf7-c1eef37a231c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.277476 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1322/2636] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:03 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:03.282056 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95291af4-e4a1-4e34-bbf7-c1eef37a231c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.300257 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95291af4-e4a1-4e34-bbf7-c1eef37a231c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.305834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639fb26-f7af-41e0-8246-d95b8724ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.306574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639fb26-f7af-41e0-8246-d95b8724ced7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.306937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639fb26-f7af-41e0-8246-d95b8724ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.307271 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639fb26-f7af-41e0-8246-d95b8724ced7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.307998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639fb26-f7af-41e0-8246-d95b8724ced7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.308731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bcade-ebfa-4054-bc8c-3757e3f97629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.309383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bcade-ebfa-4054-bc8c-3757e3f97629 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.309699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bcade-ebfa-4054-bc8c-3757e3f97629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.309872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639fb26-f7af-41e0-8246-d95b8724ced7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.309974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bcade-ebfa-4054-bc8c-3757e3f97629 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.310607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bcade-ebfa-4054-bc8c-3757e3f97629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.312529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bcade-ebfa-4054-bc8c-3757e3f97629 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.324477 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1323/2637] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:03.325054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62f68-b12c-498a-af48-da64d1f752c0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.325343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62f68-b12c-498a-af48-da64d1f752c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.325655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e62f68-b12c-498a-af48-da64d1f752c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.345126 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1315/2638] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:03 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:03.347184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bcade-ebfa-4054-bc8c-3757e3f97629 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.357475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-65e62f68-b12c-498a-af48-da64d1f752c0 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:03.358901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-65e62f68-b12c-498a-af48-da64d1f752c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:03.360468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bcade-ebfa-4054-bc8c-3757e3f97629 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.367179 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d286f-571b-42c7-b091-72bbb70fe186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.367934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d286f-571b-42c7-b091-72bbb70fe186 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.368220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d286f-571b-42c7-b091-72bbb70fe186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.368604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d286f-571b-42c7-b091-72bbb70fe186 admin admin] PATH_INFO: `/v3/projects/1357962577ed4717b942f81f7bebe7a4/users/06b7b58c08e441e397770104cbd2bc2a/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.387192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d286f-571b-42c7-b091-72bbb70fe186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1357962577ed4717b942f81f7bebe7a4, user_id=06b7b58c08e441e397770104cbd2bc2a, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.393224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d286f-571b-42c7-b091-72bbb70fe186 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.414382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1316/2639] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:03 2022] PUT /identity/v3/projects/1357962577ed4717b942f81f7bebe7a4/users/06b7b58c08e441e397770104cbd2bc2a/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:03.418019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d286f-571b-42c7-b091-72bbb70fe186 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.427199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1324/2640] 10.222.0.58 () {58 vars in 967 bytes} [Mon Oct 3 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:03.434417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e62f68-b12c-498a-af48-da64d1f752c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.453242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e62f68-b12c-498a-af48-da64d1f752c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.467876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:03.491188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d286f-571b-42c7-b091-72bbb70fe186 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.502911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.503586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.503877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.504158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.504825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.506617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.518356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1317/2641] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:03.519237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.521955 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:03.528046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fafc55-a8de-4858-9a10-ca16f884dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.528680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fafc55-a8de-4858-9a10-ca16f884dde4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.528969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fafc55-a8de-4858-9a10-ca16f884dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.529253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fafc55-a8de-4858-9a10-ca16f884dde4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.529954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fafc55-a8de-4858-9a10-ca16f884dde4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.531624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fafc55-a8de-4858-9a10-ca16f884dde4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.543963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1325/2642] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:03.544855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fafc55-a8de-4858-9a10-ca16f884dde4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.546830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:03.557730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fafc55-a8de-4858-9a10-ca16f884dde4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.563903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91116537-9a86-47bc-aff7-dac63834e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.565412 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91116537-9a86-47bc-aff7-dac63834e2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.565922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91116537-9a86-47bc-aff7-dac63834e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.566403 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91116537-9a86-47bc-aff7-dac63834e2d9 admin admin] PATH_INFO: `/v3/projects/931bf9606bed42ddadd0c8ffc9fbac15/users/a3881d67cec344ac974c1f42e8ef2ed9/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.587285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91116537-9a86-47bc-aff7-dac63834e2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931bf9606bed42ddadd0c8ffc9fbac15, user_id=a3881d67cec344ac974c1f42e8ef2ed9, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.592291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91116537-9a86-47bc-aff7-dac63834e2d9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.603782 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c72ee-ed32-4532-9d35-9b8382dff1ac admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.609720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1326/2643] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:03 2022] PUT /identity/v3/projects/931bf9606bed42ddadd0c8ffc9fbac15/users/a3881d67cec344ac974c1f42e8ef2ed9/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:03.610608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91116537-9a86-47bc-aff7-dac63834e2d9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.687586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91116537-9a86-47bc-aff7-dac63834e2d9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.692824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbea53-9d39-41db-8887-cf274c4f3aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.693675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766b491-aac3-48a7-a22f-908b174c9b68 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.693861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbea53-9d39-41db-8887-cf274c4f3aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.693954 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbea53-9d39-41db-8887-cf274c4f3aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.694237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbea53-9d39-41db-8887-cf274c4f3aa8 admin admin] PATH_INFO: `/v3/projects/1357962577ed4717b942f81f7bebe7a4/users/06b7b58c08e441e397770104cbd2bc2a/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.694741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766b491-aac3-48a7-a22f-908b174c9b68 service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.695257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766b491-aac3-48a7-a22f-908b174c9b68 service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.695773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766b491-aac3-48a7-a22f-908b174c9b68 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.696251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbea53-9d39-41db-8887-cf274c4f3aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1357962577ed4717b942f81f7bebe7a4, user_id=06b7b58c08e441e397770104cbd2bc2a, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.698970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbea53-9d39-41db-8887-cf274c4f3aa8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.711554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1327/2644] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:03 2022] PUT /identity/v3/projects/1357962577ed4717b942f81f7bebe7a4/users/06b7b58c08e441e397770104cbd2bc2a/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:03.712423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbea53-9d39-41db-8887-cf274c4f3aa8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.774141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766b491-aac3-48a7-a22f-908b174c9b68 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.776484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766b491-aac3-48a7-a22f-908b174c9b68 service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.788690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1318/2645] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:40:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:40:03.790237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8719472-4860-4a7b-872d-882c8fab9b8b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.790535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8719472-4860-4a7b-872d-882c8fab9b8b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.791127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8719472-4860-4a7b-872d-882c8fab9b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.809149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbea53-9d39-41db-8887-cf274c4f3aa8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.818471 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd204b65-3ed1-4d62-838c-8c1f45f3b06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.820171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204b65-3ed1-4d62-838c-8c1f45f3b06f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.820680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204b65-3ed1-4d62-838c-8c1f45f3b06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.821154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd204b65-3ed1-4d62-838c-8c1f45f3b06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.822303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd204b65-3ed1-4d62-838c-8c1f45f3b06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.825733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd204b65-3ed1-4d62-838c-8c1f45f3b06f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.832857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a8719472-4860-4a7b-872d-882c8fab9b8b None None] MFA Rules not processed for user `06b7b58c08e441e397770104cbd2bc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:03.834492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a8719472-4860-4a7b-872d-882c8fab9b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:03.850420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1328/2646] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:03.856002 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd204b65-3ed1-4d62-838c-8c1f45f3b06f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.872577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd204b65-3ed1-4d62-838c-8c1f45f3b06f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:03.884742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af439a-c11c-4283-b0e3-fb425a7f336a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:03.885582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af439a-c11c-4283-b0e3-fb425a7f336a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.885779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af439a-c11c-4283-b0e3-fb425a7f336a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.886124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af439a-c11c-4283-b0e3-fb425a7f336a admin admin] PATH_INFO: `/v3/projects/931bf9606bed42ddadd0c8ffc9fbac15/users/a3881d67cec344ac974c1f42e8ef2ed9/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.888559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af439a-c11c-4283-b0e3-fb425a7f336a admin admin] RBAC: Authorizing `identity:create_grant(project_id=931bf9606bed42ddadd0c8ffc9fbac15, user_id=a3881d67cec344ac974c1f42e8ef2ed9, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:03.891909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af439a-c11c-4283-b0e3-fb425a7f336a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:03.905911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1329/2647] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:03 2022] PUT /identity/v3/projects/931bf9606bed42ddadd0c8ffc9fbac15/users/a3881d67cec344ac974c1f42e8ef2ed9/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:03.906212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:03.918374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa9b2-fba5-4295-a4eb-b6415966ff16 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:03.918509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa9b2-fba5-4295-a4eb-b6415966ff16 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:03.918813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fa9b2-fba5-4295-a4eb-b6415966ff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:03.930156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:03.954922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1319/2648] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:03.966825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cc5fa9b2-fba5-4295-a4eb-b6415966ff16 None None] MFA Rules not processed for user `a3881d67cec344ac974c1f42e8ef2ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:03.969297 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cc5fa9b2-fba5-4295-a4eb-b6415966ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:03.976470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8719472-4860-4a7b-872d-882c8fab9b8b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:03.978171 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a8719472-4860-4a7b-872d-882c8fab9b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:04.065510 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8719472-4860-4a7b-872d-882c8fab9b8b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:04.096558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1330/2649] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:04.112089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fa9b2-fba5-4295-a4eb-b6415966ff16 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:04.114139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cc5fa9b2-fba5-4295-a4eb-b6415966ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:04.173503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a4172-a2de-42cf-aadc-184b730ba0b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:04.173503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a4172-a2de-42cf-aadc-184b730ba0b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:04.173503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a4172-a2de-42cf-aadc-184b730ba0b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:04.173503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a4172-a2de-42cf-aadc-184b730ba0b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:04.210750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fa9b2-fba5-4295-a4eb-b6415966ff16 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:04.215342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba894-67cb-4d21-b9c3-5bd91134b9cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:04.216034 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba894-67cb-4d21-b9c3-5bd91134b9cc service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:04.216380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba894-67cb-4d21-b9c3-5bd91134b9cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:04.216809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dba894-67cb-4d21-b9c3-5bd91134b9cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:04.262237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a4172-a2de-42cf-aadc-184b730ba0b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:04.263943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a4172-a2de-42cf-aadc-184b730ba0b5 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:04.277787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1320/2650] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:40:04.300038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dba894-67cb-4d21-b9c3-5bd91134b9cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:04.301685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dba894-67cb-4d21-b9c3-5bd91134b9cc service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:04.312699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1331/2651] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:06.037293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a4172-a2de-42cf-aadc-184b730ba0b5 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:06.048302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a4172-a2de-42cf-aadc-184b730ba0b5 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:06.054403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f31b-9b33-41ed-be29-bb88591d83cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:06.055115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f31b-9b33-41ed-be29-bb88591d83cc service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:06.055496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f31b-9b33-41ed-be29-bb88591d83cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:06.055928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8693f31b-9b33-41ed-be29-bb88591d83cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:06.166434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:06.168689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693f31b-9b33-41ed-be29-bb88591d83cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:06.170959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8693f31b-9b33-41ed-be29-bb88591d83cc service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:06.186109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1321/2652] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:40:06.929868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba894-67cb-4d21-b9c3-5bd91134b9cc service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:06.940415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dba894-67cb-4d21-b9c3-5bd91134b9cc service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:06.944623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b032bc-1a6a-46eb-afd6-5b0aab2ab73f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:06.945243 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b032bc-1a6a-46eb-afd6-5b0aab2ab73f service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:06.945559 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b032bc-1a6a-46eb-afd6-5b0aab2ab73f service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:06.945845 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b032bc-1a6a-46eb-afd6-5b0aab2ab73f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:06.956978 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b032bc-1a6a-46eb-afd6-5b0aab2ab73f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:06.958965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b032bc-1a6a-46eb-afd6-5b0aab2ab73f service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:06.969020 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1332/2653] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:09.017977 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f31b-9b33-41ed-be29-bb88591d83cc service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:09.027102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8693f31b-9b33-41ed-be29-bb88591d83cc service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:09.030840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4feec-af98-4bb5-9fd5-274709ed0b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:09.031454 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4feec-af98-4bb5-9fd5-274709ed0b54 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:09.031738 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4feec-af98-4bb5-9fd5-274709ed0b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:09.032016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4feec-af98-4bb5-9fd5-274709ed0b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:09.103333 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4feec-af98-4bb5-9fd5-274709ed0b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:09.104803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4feec-af98-4bb5-9fd5-274709ed0b54 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:09.123025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1322/2654] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:10.989498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b032bc-1a6a-46eb-afd6-5b0aab2ab73f service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:11.029492 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:11.062980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b032bc-1a6a-46eb-afd6-5b0aab2ab73f service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:11.068269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:11.068911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:11.069198 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:11.069509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] PATH_INFO: `/v3/users/4c89afe07c794464afc8606ec2a3bcfe` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:11.070670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c89afe07c794464afc8606ec2a3bcfe)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:11.072616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:11.127353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c89afe07c794464afc8606ec2a3bcfe'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.141023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c89afe07c794464afc8606ec2a3bcfe'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.141769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c89afe07c794464afc8606ec2a3bcfe'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.151142 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c89afe07c794464afc8606ec2a3bcfe'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.160370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c89afe07c794464afc8606ec2a3bcfe'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.166646 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1333/2655] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:10 2022] DELETE /identity/v3/users/4c89afe07c794464afc8606ec2a3bcfe => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:11.171101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f47ea-3350-44f4-8073-41550f183610 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:11.171600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f47ea-3350-44f4-8073-41550f183610 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:11.171743 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f47ea-3350-44f4-8073-41550f183610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:11.191213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0f2f47ea-3350-44f4-8073-41550f183610 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:11.192386 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0f2f47ea-3350-44f4-8073-41550f183610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:11.254331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1323/2656] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:11.264199 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:11.265757 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:11.328234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f3f5-fd06-40c9-b7c5-e8c2c5dd84cd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:11.332957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c89166-36d1-4c16-9bc4-48f0aceb04cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:11.333638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c89166-36d1-4c16-9bc4-48f0aceb04cd service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:11.333930 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c89166-36d1-4c16-9bc4-48f0aceb04cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:11.334203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c89166-36d1-4c16-9bc4-48f0aceb04cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:11.344545 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c89166-36d1-4c16-9bc4-48f0aceb04cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:11.346848 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c89166-36d1-4c16-9bc4-48f0aceb04cd service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:11.357647 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1334/2657] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:40:11.672728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f47ea-3350-44f4-8073-41550f183610 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:11.674440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0f2f47ea-3350-44f4-8073-41550f183610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:11.692648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f47ea-3350-44f4-8073-41550f183610 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:11.698083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:11.698586 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:11.698870 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:11.699166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] PATH_INFO: `/v3/projects/1745da99c8b74b04a04fbc70e603b24e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:11.700459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1745da99c8b74b04a04fbc70e603b24e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:11.702400 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:11.765812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1745da99c8b74b04a04fbc70e603b24e', 'request_id': 'req-bceb4f62-d68a-493c-9a34-01c72d1ab06c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.773195 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1745da99c8b74b04a04fbc70e603b24e', 'request_id': 'req-bceb4f62-d68a-493c-9a34-01c72d1ab06c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.779970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1745da99c8b74b04a04fbc70e603b24e', 'request_id': 'req-bceb4f62-d68a-493c-9a34-01c72d1ab06c'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.794511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] The token cache is being invalidate because project 1745da99c8b74b04a04fbc70e603b24e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:11.794821 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.795494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.797057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1324/2658] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:11 2022] DELETE /identity/v3/projects/1745da99c8b74b04a04fbc70e603b24e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:11.801783 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c89166-36d1-4c16-9bc4-48f0aceb04cd service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:11.813414 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c89166-36d1-4c16-9bc4-48f0aceb04cd service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:11.877831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:11.890151 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:11.891038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:11.891527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:11.891942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] PATH_INFO: `/v3/users/c494c20c755246dc9c4b11ae85c5e13d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:11.893977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c494c20c755246dc9c4b11ae85c5e13d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:11.896043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:11.946901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c494c20c755246dc9c4b11ae85c5e13d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.960765 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c494c20c755246dc9c4b11ae85c5e13d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.961529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c494c20c755246dc9c4b11ae85c5e13d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.969956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c494c20c755246dc9c4b11ae85c5e13d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.977966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c494c20c755246dc9c4b11ae85c5e13d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:11.984014 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1335/2659] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:11 2022] DELETE /identity/v3/users/c494c20c755246dc9c4b11ae85c5e13d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:12.260594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:12.262094 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:12.276264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb4f62-d68a-493c-9a34-01c72d1ab06c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:12.287366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:12.338799 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:12.339371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:12.339658 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:12.339936 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] PATH_INFO: `/v3/projects/f965eb6d51f143c6a099deff7814ca32` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:12.341069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f965eb6d51f143c6a099deff7814ca32)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:12.343052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:12.390208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f965eb6d51f143c6a099deff7814ca32', 'request_id': 'req-b91e6b6a-e768-437b-b2fc-8b04339bc69d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.396881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f965eb6d51f143c6a099deff7814ca32', 'request_id': 'req-b91e6b6a-e768-437b-b2fc-8b04339bc69d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.402570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f965eb6d51f143c6a099deff7814ca32', 'request_id': 'req-b91e6b6a-e768-437b-b2fc-8b04339bc69d'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.415127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] The token cache is being invalidate because project f965eb6d51f143c6a099deff7814ca32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:12.415422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.416047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.417579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1325/2660] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:12 2022] DELETE /identity/v3/projects/f965eb6d51f143c6a099deff7814ca32 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:12.421610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:12.423109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:12.436364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ede34-2dc9-4b66-b2fc-d9c3910ed81a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:12.494667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:12.495279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:12.495563 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:12.495841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] PATH_INFO: `/v3/users/a3881d67cec344ac974c1f42e8ef2ed9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:12.496981 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3881d67cec344ac974c1f42e8ef2ed9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:12.498936 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:12.552813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3881d67cec344ac974c1f42e8ef2ed9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.565264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3881d67cec344ac974c1f42e8ef2ed9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.565994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3881d67cec344ac974c1f42e8ef2ed9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.573574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3881d67cec344ac974c1f42e8ef2ed9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.575915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:12.584107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6451a911-0d1a-4f44-9e54-ea547382f645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3881d67cec344ac974c1f42e8ef2ed9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:12.590665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1336/2661] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:12 2022] DELETE /identity/v3/users/a3881d67cec344ac974c1f42e8ef2ed9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:12.880943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:12.882445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:12.898227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e6b6a-e768-437b-b2fc-8b04339bc69d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:12.961733 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:12.962464 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:12.962758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:12.963041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] PATH_INFO: `/v3/projects/931bf9606bed42ddadd0c8ffc9fbac15` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:12.964197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=931bf9606bed42ddadd0c8ffc9fbac15)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:12.966189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.023958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:13.030706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931bf9606bed42ddadd0c8ffc9fbac15', 'request_id': 'req-80f380c5-d662-4bf0-866f-05a2cfa7cb43'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.041913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931bf9606bed42ddadd0c8ffc9fbac15', 'request_id': 'req-80f380c5-d662-4bf0-866f-05a2cfa7cb43'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.051375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931bf9606bed42ddadd0c8ffc9fbac15', 'request_id': 'req-80f380c5-d662-4bf0-866f-05a2cfa7cb43'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.070654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] The token cache is being invalidate because project 931bf9606bed42ddadd0c8ffc9fbac15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:13.070987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.071699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-80f380c5-d662-4bf0-866f-05a2cfa7cb43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.073561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1326/2662] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:12 2022] DELETE /identity/v3/projects/931bf9606bed42ddadd0c8ffc9fbac15 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:13.079009 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2358e2a-5c87-41ac-9b51-0753acd0e06c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.079422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2358e2a-5c87-41ac-9b51-0753acd0e06c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.079880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2358e2a-5c87-41ac-9b51-0753acd0e06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.102971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b2358e2a-5c87-41ac-9b51-0753acd0e06c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:13.104673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b2358e2a-5c87-41ac-9b51-0753acd0e06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:13.181806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1337/2663] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:13.250808 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491630ff-b349-4f3e-8967-a96e932df411 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.251133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491630ff-b349-4f3e-8967-a96e932df411 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.251495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491630ff-b349-4f3e-8967-a96e932df411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.273236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-491630ff-b349-4f3e-8967-a96e932df411 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:13.274511 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-491630ff-b349-4f3e-8967-a96e932df411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:13.282584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1327/2664] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:13.287039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2358e2a-5c87-41ac-9b51-0753acd0e06c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:13.296802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2358e2a-5c87-41ac-9b51-0753acd0e06c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:13.301661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:13.302301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.302591 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.302872 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.303535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:40:13.303847 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:13.305479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.317976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1338/2665] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:40:13.322311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491630ff-b349-4f3e-8967-a96e932df411 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:13.332566 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491630ff-b349-4f3e-8967-a96e932df411 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:13.337865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a59b5-6d11-48f5-af4c-5bf1d61b7e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:13.339079 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a59b5-6d11-48f5-af4c-5bf1d61b7e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.339079 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a59b5-6d11-48f5-af4c-5bf1d61b7e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.339079 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a59b5-6d11-48f5-af4c-5bf1d61b7e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.339699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a59b5-6d11-48f5-af4c-5bf1d61b7e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:13.341664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a59b5-6d11-48f5-af4c-5bf1d61b7e8b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.369051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1328/2666] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:13 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:13.373260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:13.384250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c26880-e2eb-45f7-892e-c6ee892e0d3f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:13.441287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414fec7-0e16-41a6-99d6-3110c37a13f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:13.441988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414fec7-0e16-41a6-99d6-3110c37a13f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.442197 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414fec7-0e16-41a6-99d6-3110c37a13f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.442471 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414fec7-0e16-41a6-99d6-3110c37a13f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.443115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414fec7-0e16-41a6-99d6-3110c37a13f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:13.445019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414fec7-0e16-41a6-99d6-3110c37a13f3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.466750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1339/2667] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:13 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:13.471050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a59b5-6d11-48f5-af4c-5bf1d61b7e8b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:13.481693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a59b5-6d11-48f5-af4c-5bf1d61b7e8b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:13.486864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd7471-bab5-408c-a8d2-40473e7bb471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:13.487488 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7471-bab5-408c-a8d2-40473e7bb471 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.487750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7471-bab5-408c-a8d2-40473e7bb471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.488023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd7471-bab5-408c-a8d2-40473e7bb471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.488662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd7471-bab5-408c-a8d2-40473e7bb471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:13.490304 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd7471-bab5-408c-a8d2-40473e7bb471 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.501331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1329/2668] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:13.505736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414fec7-0e16-41a6-99d6-3110c37a13f3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:13.516429 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414fec7-0e16-41a6-99d6-3110c37a13f3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:13.521989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29085c0f-4523-4d30-89b0-f249d39ac1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:13.522597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085c0f-4523-4d30-89b0-f249d39ac1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.522880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085c0f-4523-4d30-89b0-f249d39ac1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.523158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29085c0f-4523-4d30-89b0-f249d39ac1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.523806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29085c0f-4523-4d30-89b0-f249d39ac1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:13.525413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29085c0f-4523-4d30-89b0-f249d39ac1e9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.537288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1340/2669] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:13.541647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd7471-bab5-408c-a8d2-40473e7bb471 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:13.551787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd7471-bab5-408c-a8d2-40473e7bb471 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:13.556380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:13.557026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.557308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.557623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] PATH_INFO: `/v3/projects/e938fc9b01cb4a008f26cc5cca042a3d/users/544a68310a5f4b388b98c1c5cae75b88/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.572932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e938fc9b01cb4a008f26cc5cca042a3d, user_id=544a68310a5f4b388b98c1c5cae75b88, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:13.575542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.589284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1330/2670] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:13 2022] PUT /identity/v3/projects/e938fc9b01cb4a008f26cc5cca042a3d/users/544a68310a5f4b388b98c1c5cae75b88/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:13.600074 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0a85e-77c8-4f6b-8daa-bd0cd777b5bb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.600480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0a85e-77c8-4f6b-8daa-bd0cd777b5bb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.600769 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0a85e-77c8-4f6b-8daa-bd0cd777b5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.629886 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d4d0a85e-77c8-4f6b-8daa-bd0cd777b5bb None None] MFA Rules not processed for user `544a68310a5f4b388b98c1c5cae75b88`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:13.631147 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d4d0a85e-77c8-4f6b-8daa-bd0cd777b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:13.677512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:13.697791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1341/2671] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:13.709796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:13.711512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:13.773629 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2469e2c-a1b3-4baf-b430-d9708e4aef72 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:13.777407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a9587a-5fbe-4ae5-af30-99267ada95f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:13.778051 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9587a-5fbe-4ae5-af30-99267ada95f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.778334 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9587a-5fbe-4ae5-af30-99267ada95f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.778610 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a9587a-5fbe-4ae5-af30-99267ada95f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.787465 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9587a-5fbe-4ae5-af30-99267ada95f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:13.788913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a9587a-5fbe-4ae5-af30-99267ada95f9 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.797770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1331/2672] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:13.824802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0a85e-77c8-4f6b-8daa-bd0cd777b5bb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:13.897598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0a85e-77c8-4f6b-8daa-bd0cd777b5bb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:13.902428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:13.903081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:13.903484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:13.903775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] PATH_INFO: `/v3/users/544a68310a5f4b388b98c1c5cae75b88` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:13.904966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=544a68310a5f4b388b98c1c5cae75b88)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:13.906965 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:13.961194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544a68310a5f4b388b98c1c5cae75b88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.975138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544a68310a5f4b388b98c1c5cae75b88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.975893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544a68310a5f4b388b98c1c5cae75b88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.984386 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544a68310a5f4b388b98c1c5cae75b88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:13.993683 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544a68310a5f4b388b98c1c5cae75b88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.000346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1342/2673] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:13 2022] DELETE /identity/v3/users/544a68310a5f4b388b98c1c5cae75b88 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:14.005558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f8a84-00ab-4d78-b7c0-91a7cbd44b9f None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.005848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f8a84-00ab-4d78-b7c0-91a7cbd44b9f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.006125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f8a84-00ab-4d78-b7c0-91a7cbd44b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.026712 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-056f8a84-00ab-4d78-b7c0-91a7cbd44b9f None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:14.028017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-056f8a84-00ab-4d78-b7c0-91a7cbd44b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.091293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1332/2674] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:14.102891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.104510 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.104785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f8a84-00ab-4d78-b7c0-91a7cbd44b9f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.106147 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-056f8a84-00ab-4d78-b7c0-91a7cbd44b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.119710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b6357-da7e-4a12-9396-3572aef649d3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:14.121731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:14.126060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:14.127315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.127611 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.127959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] PATH_INFO: `/v3/users/4bba3c927c18493eb558e1823ffa78ce` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.129115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bba3c927c18493eb558e1823ffa78ce)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:14.131144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:14.171152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f8a84-00ab-4d78-b7c0-91a7cbd44b9f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:14.176312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541bdb0-4365-40c7-ac10-7909f8df90f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:14.176939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541bdb0-4365-40c7-ac10-7909f8df90f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.177225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541bdb0-4365-40c7-ac10-7909f8df90f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.177559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541bdb0-4365-40c7-ac10-7909f8df90f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.183359 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bba3c927c18493eb558e1823ffa78ce'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.187302 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541bdb0-4365-40c7-ac10-7909f8df90f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:14.188763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541bdb0-4365-40c7-ac10-7909f8df90f2 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:14.194473 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:14.196716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bba3c927c18493eb558e1823ffa78ce'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.197687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bba3c927c18493eb558e1823ffa78ce'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.199494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1333/2675] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:14.207501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bba3c927c18493eb558e1823ffa78ce'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.214977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bba3c927c18493eb558e1823ffa78ce'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.221957 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1343/2676] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:14 2022] DELETE /identity/v3/users/4bba3c927c18493eb558e1823ffa78ce => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:14.226681 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dfadb-e0f7-4d31-9aa4-e75891eddaf9 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.226981 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dfadb-e0f7-4d31-9aa4-e75891eddaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.227249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dfadb-e0f7-4d31-9aa4-e75891eddaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.248847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4c3dfadb-e0f7-4d31-9aa4-e75891eddaf9 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:14.250399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4c3dfadb-e0f7-4d31-9aa4-e75891eddaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.328431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1334/2677] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:14.338753 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.340229 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.413732 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933c281-87f7-42b7-87e1-e7a4b8989221 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:14.417588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2ec49-dd36-41f8-afa7-1dcf4e96e8bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:14.418213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2ec49-dd36-41f8-afa7-1dcf4e96e8bd service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.418497 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2ec49-dd36-41f8-afa7-1dcf4e96e8bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.418896 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2ec49-dd36-41f8-afa7-1dcf4e96e8bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.438644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2ec49-dd36-41f8-afa7-1dcf4e96e8bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:14.440173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2ec49-dd36-41f8-afa7-1dcf4e96e8bd service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:14.450322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1344/2678] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:40:14.554587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3dfadb-e0f7-4d31-9aa4-e75891eddaf9 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.556293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4c3dfadb-e0f7-4d31-9aa4-e75891eddaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.573019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3dfadb-e0f7-4d31-9aa4-e75891eddaf9 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:14.579767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:14.581115 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.581429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.581722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] PATH_INFO: `/v3/projects/e938fc9b01cb4a008f26cc5cca042a3d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.582980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e938fc9b01cb4a008f26cc5cca042a3d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:14.584872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:14.638205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e938fc9b01cb4a008f26cc5cca042a3d', 'request_id': 'req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.650157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e938fc9b01cb4a008f26cc5cca042a3d', 'request_id': 'req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.656812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e938fc9b01cb4a008f26cc5cca042a3d', 'request_id': 'req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.671116 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] The token cache is being invalidate because project e938fc9b01cb4a008f26cc5cca042a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:14.671419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.672078 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-9683ed66-ae0b-4f68-a640-6fb1c22ed8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.673826 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1335/2679] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:14 2022] DELETE /identity/v3/projects/e938fc9b01cb4a008f26cc5cca042a3d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:14.678788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf4f41-2621-465c-826a-e995cba7897d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.679077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf4f41-2621-465c-826a-e995cba7897d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.679415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf4f41-2621-465c-826a-e995cba7897d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.702368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dbcf4f41-2621-465c-826a-e995cba7897d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:14.703603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dbcf4f41-2621-465c-826a-e995cba7897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.769331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1345/2680] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:14.778236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dd293-0620-4bde-8aee-75fa81222b10 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.778236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dd293-0620-4bde-8aee-75fa81222b10 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.778236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649dd293-0620-4bde-8aee-75fa81222b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.779083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf4f41-2621-465c-826a-e995cba7897d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.780538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dbcf4f41-2621-465c-826a-e995cba7897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.797220 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf4f41-2621-465c-826a-e995cba7897d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:14.802861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:14.803618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.803903 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.804024 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-649dd293-0620-4bde-8aee-75fa81222b10 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:14.804173 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] PATH_INFO: `/v3/projects/d2e1596319e74fbe909e1be08c59b683` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.805265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-649dd293-0620-4bde-8aee-75fa81222b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.805388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e1596319e74fbe909e1be08c59b683)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:14.807451 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:14.815329 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1336/2681] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:14.819286 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dd293-0620-4bde-8aee-75fa81222b10 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.828866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649dd293-0620-4bde-8aee-75fa81222b10 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:14.834873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:14.835536 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.835796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.836069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.836726 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:40:14.837029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:14.838708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:14.858061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1337/2682] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:40:14.863502 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.866010 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e1596319e74fbe909e1be08c59b683', 'request_id': 'req-97e2f6f9-0135-4165-bf31-b99d952c16cf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.874561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4b048-11fc-4b29-8ef6-f34d6996fd4d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:14.875552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e1596319e74fbe909e1be08c59b683', 'request_id': 'req-97e2f6f9-0135-4165-bf31-b99d952c16cf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.889801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e1596319e74fbe909e1be08c59b683', 'request_id': 'req-97e2f6f9-0135-4165-bf31-b99d952c16cf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.899092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:14.904681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] The token cache is being invalidate because project d2e1596319e74fbe909e1be08c59b683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:14.904980 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.905704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:14.907417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1346/2683] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:14 2022] DELETE /identity/v3/projects/d2e1596319e74fbe909e1be08c59b683 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:14.911466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.912868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:14.919038 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:14.944992 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:14.945671 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:14.945962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:14.946407 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:14.947063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:14.948986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:14.979975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1338/2684] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:14 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:14.984240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:14.985860 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2f6f9-0135-4165-bf31-b99d952c16cf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:14.986509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:15.051348 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.052000 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.052283 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.052564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] PATH_INFO: `/v3/users/06b7b58c08e441e397770104cbd2bc2a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.053776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b7b58c08e441e397770104cbd2bc2a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.055715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.058494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93436d-444a-460b-9d90-2ac1668ca927 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:15.063634 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a248-6f51-42a5-8b02-d3c35f759a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.064305 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a248-6f51-42a5-8b02-d3c35f759a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.064584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a248-6f51-42a5-8b02-d3c35f759a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.064860 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7a248-6f51-42a5-8b02-d3c35f759a1e admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.065534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7a248-6f51-42a5-8b02-d3c35f759a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.067467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7a248-6f51-42a5-8b02-d3c35f759a1e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.092775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1339/2685] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:14 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:15.097894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a248-6f51-42a5-8b02-d3c35f759a1e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:15.108411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7a248-6f51-42a5-8b02-d3c35f759a1e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:15.109442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b7b58c08e441e397770104cbd2bc2a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.113828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.114550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.114915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.115197 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.115865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.117553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.125091 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b7b58c08e441e397770104cbd2bc2a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.125911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b7b58c08e441e397770104cbd2bc2a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.137526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1340/2686] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:15.139830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:15.143340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:15.144748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b7b58c08e441e397770104cbd2bc2a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.157171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b7b58c08e441e397770104cbd2bc2a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.157878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cf309-0295-4dd0-a646-1d9f3b12e1f1 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:15.163986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844140a-c9f7-44c9-a981-53cae7122f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.164626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844140a-c9f7-44c9-a981-53cae7122f9f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.164910 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844140a-c9f7-44c9-a981-53cae7122f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.165188 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844140a-c9f7-44c9-a981-53cae7122f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.165900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844140a-c9f7-44c9-a981-53cae7122f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.167079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1347/2687] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:14 2022] DELETE /identity/v3/users/06b7b58c08e441e397770104cbd2bc2a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:15.167577 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844140a-c9f7-44c9-a981-53cae7122f9f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.185002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1341/2688] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:15.189694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:15.230415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:15.268329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf398e78-4daf-4e21-97bf-4977b99f409e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:15.273432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.274284 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.274574 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.274859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] PATH_INFO: `/v3/projects/0e005cbca8874c76bf8ad8f5d336b189/users/f548ccf8c5a14b108aa8b851560139d0/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.296556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e005cbca8874c76bf8ad8f5d336b189, user_id=f548ccf8c5a14b108aa8b851560139d0, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.300367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.318564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1348/2689] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:15 2022] PUT /identity/v3/projects/0e005cbca8874c76bf8ad8f5d336b189/users/f548ccf8c5a14b108aa8b851560139d0/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:15.323885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf196e-d485-45d8-ba06-0d96a7b4eb89 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.324196 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf196e-d485-45d8-ba06-0d96a7b4eb89 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.324529 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf196e-d485-45d8-ba06-0d96a7b4eb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.356689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-efcf196e-d485-45d8-ba06-0d96a7b4eb89 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:15.360101 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-efcf196e-d485-45d8-ba06-0d96a7b4eb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:15.436941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1342/2690] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:15.448313 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:15.450178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:15.496330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf196e-d485-45d8-ba06-0d96a7b4eb89 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:15.498169 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-efcf196e-d485-45d8-ba06-0d96a7b4eb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:15.524052 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf196e-d485-45d8-ba06-0d96a7b4eb89 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:15.532303 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.533585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.534041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.534209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] PATH_INFO: `/v3/projects/1357962577ed4717b942f81f7bebe7a4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.535368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1357962577ed4717b942f81f7bebe7a4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.537382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.544966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c63351-357c-4082-9f86-397fd563a5b5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:15.550360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:15.553503 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1323ed-8769-4709-91e9-033b18992eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.554538 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1323ed-8769-4709-91e9-033b18992eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.555044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1323ed-8769-4709-91e9-033b18992eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.555663 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1323ed-8769-4709-91e9-033b18992eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.569743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1323ed-8769-4709-91e9-033b18992eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.571955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1323ed-8769-4709-91e9-033b18992eb1 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.588479 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1349/2691] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:40:15.613926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1357962577ed4717b942f81f7bebe7a4', 'request_id': 'req-a8ac1efe-2030-4005-93b7-aa649421280f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.622280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1357962577ed4717b942f81f7bebe7a4', 'request_id': 'req-a8ac1efe-2030-4005-93b7-aa649421280f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.632656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1357962577ed4717b942f81f7bebe7a4', 'request_id': 'req-a8ac1efe-2030-4005-93b7-aa649421280f'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.650513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] The token cache is being invalidate because project 1357962577ed4717b942f81f7bebe7a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:15.650885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.651654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a8ac1efe-2030-4005-93b7-aa649421280f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:15.653670 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1343/2692] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:15 2022] DELETE /identity/v3/projects/1357962577ed4717b942f81f7bebe7a4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:15.661063 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a1931-89bd-41c4-9538-e88ef22e05d4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.661364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a1931-89bd-41c4-9538-e88ef22e05d4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.661740 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a1931-89bd-41c4-9538-e88ef22e05d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.683562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-486a1931-89bd-41c4-9538-e88ef22e05d4 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:15.684840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-486a1931-89bd-41c4-9538-e88ef22e05d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:15.796711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1350/2693] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:15.806205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ffea-8a66-48b9-9dc2-ce665144455b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.806491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ffea-8a66-48b9-9dc2-ce665144455b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.806789 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213ffea-8a66-48b9-9dc2-ce665144455b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.836358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8213ffea-8a66-48b9-9dc2-ce665144455b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:15.838037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8213ffea-8a66-48b9-9dc2-ce665144455b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:15.850262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1344/2694] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:15.855597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a1931-89bd-41c4-9538-e88ef22e05d4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:15.876524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a1931-89bd-41c4-9538-e88ef22e05d4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:15.882385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.883160 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.883413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.883710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.884308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:40:15.884688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.886370 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.907335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1351/2695] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:40:15.909245 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213ffea-8a66-48b9-9dc2-ce665144455b None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:15.921406 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213ffea-8a66-48b9-9dc2-ce665144455b None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:15.927427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69463e4d-77da-4b28-8378-83bc2eeb34df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:15.928355 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69463e4d-77da-4b28-8378-83bc2eeb34df admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:15.928842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69463e4d-77da-4b28-8378-83bc2eeb34df admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:15.929385 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69463e4d-77da-4b28-8378-83bc2eeb34df admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:15.930648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69463e4d-77da-4b28-8378-83bc2eeb34df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:15.933428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69463e4d-77da-4b28-8378-83bc2eeb34df admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:15.980639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1345/2696] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:15 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:15.990922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:16.011322 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c609ec0-b8fc-494d-928d-ccd7641fb733 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:16.094440 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:16.119635 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f898da2-b11f-4f83-95e7-58eaed52e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:16.120969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f898da2-b11f-4f83-95e7-58eaed52e896 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:16.121258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f898da2-b11f-4f83-95e7-58eaed52e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:16.121566 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f898da2-b11f-4f83-95e7-58eaed52e896 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:16.122511 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f898da2-b11f-4f83-95e7-58eaed52e896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:16.127127 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f898da2-b11f-4f83-95e7-58eaed52e896 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:16.160431 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1352/2697] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:15 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:16.164683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69463e4d-77da-4b28-8378-83bc2eeb34df admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:16.177034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69463e4d-77da-4b28-8378-83bc2eeb34df admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:16.183210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35301-17c9-47c6-98a5-8da0798d0f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:16.183850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35301-17c9-47c6-98a5-8da0798d0f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:16.184264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35301-17c9-47c6-98a5-8da0798d0f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:16.184418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb35301-17c9-47c6-98a5-8da0798d0f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:16.185592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb35301-17c9-47c6-98a5-8da0798d0f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:16.187452 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb35301-17c9-47c6-98a5-8da0798d0f88 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:16.206991 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1346/2698] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:16.213231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f898da2-b11f-4f83-95e7-58eaed52e896 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:16.225050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f898da2-b11f-4f83-95e7-58eaed52e896 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:16.229948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54c24c-6b6b-4687-8662-81bcbd4c5c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:16.230578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54c24c-6b6b-4687-8662-81bcbd4c5c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:16.230861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54c24c-6b6b-4687-8662-81bcbd4c5c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:16.231184 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54c24c-6b6b-4687-8662-81bcbd4c5c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:16.231838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54c24c-6b6b-4687-8662-81bcbd4c5c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:16.233518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54c24c-6b6b-4687-8662-81bcbd4c5c59 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:16.246425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1353/2699] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:16.251019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35301-17c9-47c6-98a5-8da0798d0f88 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:16.262184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb35301-17c9-47c6-98a5-8da0798d0f88 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:16.276026 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:16.276791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:16.277144 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:16.277600 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] PATH_INFO: `/v3/projects/0e987bcc99dc4e36b13639c0cb16237e/users/1ff1626470524731bc22baafd4d86391/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:16.303294 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e987bcc99dc4e36b13639c0cb16237e, user_id=1ff1626470524731bc22baafd4d86391, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:16.306506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:16.321994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1347/2700] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:16 2022] PUT /identity/v3/projects/0e987bcc99dc4e36b13639c0cb16237e/users/1ff1626470524731bc22baafd4d86391/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:16.402341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033de786-f10a-4ea8-8023-ffbc3331aed4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:16.402637 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033de786-f10a-4ea8-8023-ffbc3331aed4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:16.402920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033de786-f10a-4ea8-8023-ffbc3331aed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:16.440150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-033de786-f10a-4ea8-8023-ffbc3331aed4 None None] MFA Rules not processed for user `1ff1626470524731bc22baafd4d86391`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:16.441279 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-033de786-f10a-4ea8-8023-ffbc3331aed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:16.556642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1354/2701] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:16.574872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:16.577161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:16.689563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ef6a6a-21d0-4c2f-bb0c-dc4068dbeed8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:16.697382 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:16.698099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:16.698381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:16.698666 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:16.713170 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:16.715423 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:16.729240 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1348/2702] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:20.680494 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7311ab-8c91-4f0a-a2c6-8da6e2a3a5bd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:20.681116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7311ab-8c91-4f0a-a2c6-8da6e2a3a5bd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:20.681116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7311ab-8c91-4f0a-a2c6-8da6e2a3a5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:20.724921 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cc7311ab-8c91-4f0a-a2c6-8da6e2a3a5bd None None] MFA Rules not processed for user `f548ccf8c5a14b108aa8b851560139d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:20.726226 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cc7311ab-8c91-4f0a-a2c6-8da6e2a3a5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:20.824500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1355/2703] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:20.829005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:20.830953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:20.884266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:20.933700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6e1cd-6fbd-4a10-bfe0-6150dfd62fb2 service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:20.938742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ce5c4-aa66-45f8-8d3c-7ea49720a946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:20.940554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ce5c4-aa66-45f8-8d3c-7ea49720a946 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:20.941258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ce5c4-aa66-45f8-8d3c-7ea49720a946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:20.941646 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ce5c4-aa66-45f8-8d3c-7ea49720a946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:20.942317 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ce5c4-aa66-45f8-8d3c-7ea49720a946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:20.944282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ce5c4-aa66-45f8-8d3c-7ea49720a946 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:20.980842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1349/2704] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:20 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:20.984905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7311ab-8c91-4f0a-a2c6-8da6e2a3a5bd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:20.997826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7311ab-8c91-4f0a-a2c6-8da6e2a3a5bd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:21.050654 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:21.072525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae302bc7-41b2-4a6a-be8f-51304a6f3c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:21.073171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae302bc7-41b2-4a6a-be8f-51304a6f3c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:21.073464 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae302bc7-41b2-4a6a-be8f-51304a6f3c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:21.073764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae302bc7-41b2-4a6a-be8f-51304a6f3c32 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:21.074435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae302bc7-41b2-4a6a-be8f-51304a6f3c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:21.076475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae302bc7-41b2-4a6a-be8f-51304a6f3c32 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:21.105324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1356/2705] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:20 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:21.109522 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ce5c4-aa66-45f8-8d3c-7ea49720a946 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:21.121283 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ce5c4-aa66-45f8-8d3c-7ea49720a946 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:21.126649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3554da4-77dc-454d-8dcf-88c5727559b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:21.127707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3554da4-77dc-454d-8dcf-88c5727559b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:21.128029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3554da4-77dc-454d-8dcf-88c5727559b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:21.128265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3554da4-77dc-454d-8dcf-88c5727559b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:21.128940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3554da4-77dc-454d-8dcf-88c5727559b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:21.130628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3554da4-77dc-454d-8dcf-88c5727559b5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:21.143824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1350/2706] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:21.149876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae302bc7-41b2-4a6a-be8f-51304a6f3c32 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:21.165577 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae302bc7-41b2-4a6a-be8f-51304a6f3c32 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:21.174995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a928aa-69af-4886-b057-571d6baa903c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:21.176251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a928aa-69af-4886-b057-571d6baa903c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:21.176549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a928aa-69af-4886-b057-571d6baa903c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:21.176835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a928aa-69af-4886-b057-571d6baa903c admin admin] PATH_INFO: `/v3/projects/ce434d4dbdbd420cbeb39478f2dbb3f0/users/66354734d9ae4e6dadbd2d59c1c4c9b1/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:21.198629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a928aa-69af-4886-b057-571d6baa903c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce434d4dbdbd420cbeb39478f2dbb3f0, user_id=66354734d9ae4e6dadbd2d59c1c4c9b1, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:21.203376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a928aa-69af-4886-b057-571d6baa903c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:21.226977 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1357/2707] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:21 2022] PUT /identity/v3/projects/ce434d4dbdbd420cbeb39478f2dbb3f0/users/66354734d9ae4e6dadbd2d59c1c4c9b1/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:21.233252 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3554da4-77dc-454d-8dcf-88c5727559b5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:21.254926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3554da4-77dc-454d-8dcf-88c5727559b5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:21.360917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317ed50-4c7f-4bb4-831b-ab1e976cb94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:21.361775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317ed50-4c7f-4bb4-831b-ab1e976cb94e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:21.362271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317ed50-4c7f-4bb4-831b-ab1e976cb94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:21.362612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317ed50-4c7f-4bb4-831b-ab1e976cb94e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:21.363350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317ed50-4c7f-4bb4-831b-ab1e976cb94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:21.365102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317ed50-4c7f-4bb4-831b-ab1e976cb94e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:21.380561 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1351/2708] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:21.385746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a928aa-69af-4886-b057-571d6baa903c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:21.399004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a928aa-69af-4886-b057-571d6baa903c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:21.404814 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2b67a-5c92-45ac-bc64-9ca656f8ce09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:21.405541 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2b67a-5c92-45ac-bc64-9ca656f8ce09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:21.405826 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2b67a-5c92-45ac-bc64-9ca656f8ce09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:21.406280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2b67a-5c92-45ac-bc64-9ca656f8ce09 admin admin] PATH_INFO: `/v3/projects/ce434d4dbdbd420cbeb39478f2dbb3f0/users/66354734d9ae4e6dadbd2d59c1c4c9b1/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:21.409690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2b67a-5c92-45ac-bc64-9ca656f8ce09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce434d4dbdbd420cbeb39478f2dbb3f0, user_id=66354734d9ae4e6dadbd2d59c1c4c9b1, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:21.412993 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2b67a-5c92-45ac-bc64-9ca656f8ce09 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:21.429900 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1358/2709] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:21 2022] PUT /identity/v3/projects/ce434d4dbdbd420cbeb39478f2dbb3f0/users/66354734d9ae4e6dadbd2d59c1c4c9b1/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:26.499922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317ed50-4c7f-4bb4-831b-ab1e976cb94e admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:26.537980 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317ed50-4c7f-4bb4-831b-ab1e976cb94e admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:26.696881 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:26.698130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:26.698607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:26.700269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] PATH_INFO: `/v3/users/1ff1626470524731bc22baafd4d86391` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:26.703064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff1626470524731bc22baafd4d86391)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:26.708102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:26.801825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff1626470524731bc22baafd4d86391'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:26.803125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff1626470524731bc22baafd4d86391'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:26.803767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff1626470524731bc22baafd4d86391'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:26.805570 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:26.821355 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff1626470524731bc22baafd4d86391'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:26.843563 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff1626470524731bc22baafd4d86391'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:26.868436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1352/2710] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:26 2022] DELETE /identity/v3/users/1ff1626470524731bc22baafd4d86391 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:26.870618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec45062-9274-4247-874d-ca3c4ca3c6b5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:26.871308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec45062-9274-4247-874d-ca3c4ca3c6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:26.871308 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec45062-9274-4247-874d-ca3c4ca3c6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:26.898033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bec45062-9274-4247-874d-ca3c4ca3c6b5 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:26.899927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bec45062-9274-4247-874d-ca3c4ca3c6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:27.012589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1359/2711] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:27.024791 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:27.026383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:27.107597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd09001-6490-4faf-b819-cd81d5342617 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:27.112675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea2d7-c667-4d7f-b5a4-2a0e6ced62ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:27.113621 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea2d7-c667-4d7f-b5a4-2a0e6ced62ca service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:27.113909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea2d7-c667-4d7f-b5a4-2a0e6ced62ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:27.114187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea2d7-c667-4d7f-b5a4-2a0e6ced62ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:27.126168 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdea2d7-c667-4d7f-b5a4-2a0e6ced62ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:27.129778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdea2d7-c667-4d7f-b5a4-2a0e6ced62ca service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:27.143713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1353/2712] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:27.757794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec45062-9274-4247-874d-ca3c4ca3c6b5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:27.763610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bec45062-9274-4247-874d-ca3c4ca3c6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:27.806854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec45062-9274-4247-874d-ca3c4ca3c6b5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:27.825759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:27.827031 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:27.827270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:27.827556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] PATH_INFO: `/v3/projects/0e987bcc99dc4e36b13639c0cb16237e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:27.829145 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e987bcc99dc4e36b13639c0cb16237e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:27.831800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:27.939262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e987bcc99dc4e36b13639c0cb16237e', 'request_id': 'req-9ede4194-f4dd-4538-8a17-0b56f3a361ee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:27.948501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e987bcc99dc4e36b13639c0cb16237e', 'request_id': 'req-9ede4194-f4dd-4538-8a17-0b56f3a361ee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:27.965820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e987bcc99dc4e36b13639c0cb16237e', 'request_id': 'req-9ede4194-f4dd-4538-8a17-0b56f3a361ee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:27.982937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] The token cache is being invalidate because project 0e987bcc99dc4e36b13639c0cb16237e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:27.983244 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:27.984036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9ede4194-f4dd-4538-8a17-0b56f3a361ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:27.986555 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1360/2713] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:27 2022] DELETE /identity/v3/projects/0e987bcc99dc4e36b13639c0cb16237e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:27.992988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd27a2-326a-4fb9-9726-477e622e717b None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:27.993893 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd27a2-326a-4fb9-9726-477e622e717b None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:27.994375 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd27a2-326a-4fb9-9726-477e622e717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.018045 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebbde0-d106-4a07-8766-8202885324ec None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.018463 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebbde0-d106-4a07-8766-8202885324ec None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.018910 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebbde0-d106-4a07-8766-8202885324ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.045612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-18dd27a2-326a-4fb9-9726-477e622e717b None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:28.048031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-18dd27a2-326a-4fb9-9726-477e622e717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:28.057353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-beebbde0-d106-4a07-8766-8202885324ec None None] MFA Rules not processed for user `66354734d9ae4e6dadbd2d59c1c4c9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:28.057830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-beebbde0-d106-4a07-8766-8202885324ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:28.068759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:28.165547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1354/2714] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:28.168720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09da5-c07c-4c30-9030-9b0e59a33e7a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.168720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09da5-c07c-4c30-9030-9b0e59a33e7a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.168720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09da5-c07c-4c30-9030-9b0e59a33e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.180481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1361/2715] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:28.205751 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4bc09da5-c07c-4c30-9030-9b0e59a33e7a None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:28.208710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebbde0-d106-4a07-8766-8202885324ec None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:28.210600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-beebbde0-d106-4a07-8766-8202885324ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:28.210982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4bc09da5-c07c-4c30-9030-9b0e59a33e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:28.227728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1355/2716] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:28.232483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc09da5-c07c-4c30-9030-9b0e59a33e7a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:28.246659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc09da5-c07c-4c30-9030-9b0e59a33e7a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:28.265431 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:28.266435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.266742 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.267158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.268087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:40:28.268714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:28.272262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:28.297327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1356/2717] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:40:28.303223 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:28.315340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918dafd-1bab-450f-bc03-7f3ef2aec191 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:28.320817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5285ee-b0c7-4ea3-90e9-e4b15c75e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:28.321493 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5285ee-b0c7-4ea3-90e9-e4b15c75e0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.321798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5285ee-b0c7-4ea3-90e9-e4b15c75e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.322083 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5285ee-b0c7-4ea3-90e9-e4b15c75e0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.322796 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5285ee-b0c7-4ea3-90e9-e4b15c75e0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:28.324909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5285ee-b0c7-4ea3-90e9-e4b15c75e0f4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:28.347805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebbde0-d106-4a07-8766-8202885324ec None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:28.355884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:28.356894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.357235 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.357830 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.360200 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:28.392650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1357/2718] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:28 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:28.400003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5285ee-b0c7-4ea3-90e9-e4b15c75e0f4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:28.438587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:28.537634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:28.540720 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:28.545998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5285ee-b0c7-4ea3-90e9-e4b15c75e0f4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:28.557626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83991ef5-99d3-4a15-a8cf-1a1f0fd239e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:28.558758 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83991ef5-99d3-4a15-a8cf-1a1f0fd239e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.559313 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83991ef5-99d3-4a15-a8cf-1a1f0fd239e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.559975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83991ef5-99d3-4a15-a8cf-1a1f0fd239e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.561830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83991ef5-99d3-4a15-a8cf-1a1f0fd239e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:28.568062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83991ef5-99d3-4a15-a8cf-1a1f0fd239e3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:28.571499 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1362/2719] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:40:28.609311 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1358/2720] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:28 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:40:28.617232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:28.640327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3fa8c-b0e2-4d75-82ba-ef770a192c34 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:28.648972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaef70-d3ab-49e8-93fd-435cb5d01684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:28.649565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaef70-d3ab-49e8-93fd-435cb5d01684 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.650052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaef70-d3ab-49e8-93fd-435cb5d01684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.650569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaef70-d3ab-49e8-93fd-435cb5d01684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.651578 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaef70-d3ab-49e8-93fd-435cb5d01684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:28.654415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eaef70-d3ab-49e8-93fd-435cb5d01684 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:28.681527 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1363/2721] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:28.686037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83991ef5-99d3-4a15-a8cf-1a1f0fd239e3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:28.703275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83991ef5-99d3-4a15-a8cf-1a1f0fd239e3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:28.713877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8a08b-655e-49c9-ba78-9dd1b27f9b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:28.714612 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a08b-655e-49c9-ba78-9dd1b27f9b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.715568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a08b-655e-49c9-ba78-9dd1b27f9b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.715568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a08b-655e-49c9-ba78-9dd1b27f9b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.716506 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8a08b-655e-49c9-ba78-9dd1b27f9b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:28.719771 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8a08b-655e-49c9-ba78-9dd1b27f9b3e admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:28.738730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1359/2722] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:40:28.744077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaef70-d3ab-49e8-93fd-435cb5d01684 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:28.759439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eaef70-d3ab-49e8-93fd-435cb5d01684 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:28.775190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:28.776303 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.776780 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.777251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] PATH_INFO: `/v3/projects/cded828d48b042e99e33885391c0c3e4/users/6ecd4219e5174bd5976877ea38c10ebf/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.808218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cded828d48b042e99e33885391c0c3e4, user_id=6ecd4219e5174bd5976877ea38c10ebf, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:28.814831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:28.837767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1364/2723] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:28 2022] PUT /identity/v3/projects/cded828d48b042e99e33885391c0c3e4/users/6ecd4219e5174bd5976877ea38c10ebf/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:28.856601 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6c3b8-9df9-40df-8977-a3311be0f866 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:28.857131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6c3b8-9df9-40df-8977-a3311be0f866 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:28.857693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6c3b8-9df9-40df-8977-a3311be0f866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:28.921785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9cb6c3b8-9df9-40df-8977-a3311be0f866 None None] MFA Rules not processed for user `6ecd4219e5174bd5976877ea38c10ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:28.924356 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9cb6c3b8-9df9-40df-8977-a3311be0f866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:29.042851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1360/2724] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:29.059334 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:29.062518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:29.073964 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:29.175571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb281f-0e23-46c2-b044-8eefc047f782 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:29.181766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73300bcb-5627-4097-9735-30f8d56c43c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:29.182744 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300bcb-5627-4097-9735-30f8d56c43c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:29.183192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300bcb-5627-4097-9735-30f8d56c43c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:29.183571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73300bcb-5627-4097-9735-30f8d56c43c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:29.238101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73300bcb-5627-4097-9735-30f8d56c43c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:29.243846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73300bcb-5627-4097-9735-30f8d56c43c8 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:29.262901 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1365/2725] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:32.239057 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6c3b8-9df9-40df-8977-a3311be0f866 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:32.346376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6c3b8-9df9-40df-8977-a3311be0f866 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:32.355436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:32.356608 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:32.357151 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:32.357730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] PATH_INFO: `/v3/users/6ecd4219e5174bd5976877ea38c10ebf` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:32.359964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ecd4219e5174bd5976877ea38c10ebf)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:32.364940 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:32.422047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:32.455813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ecd4219e5174bd5976877ea38c10ebf'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:32.457206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ecd4219e5174bd5976877ea38c10ebf'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:32.458678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ecd4219e5174bd5976877ea38c10ebf'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:32.471925 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ecd4219e5174bd5976877ea38c10ebf'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:32.505898 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ecd4219e5174bd5976877ea38c10ebf'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:32.527022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1361/2726] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:32 2022] DELETE /identity/v3/users/6ecd4219e5174bd5976877ea38c10ebf => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:32.532066 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a59a-fb46-4b09-b630-c5c92d14ec38 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:32.533715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a59a-fb46-4b09-b630-c5c92d14ec38 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:32.533715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16a59a-fb46-4b09-b630-c5c92d14ec38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:32.568944 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ae16a59a-fb46-4b09-b630-c5c92d14ec38 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:32.571104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae16a59a-fb46-4b09-b630-c5c92d14ec38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:32.674266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1366/2727] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:32.688656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:32.692635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:32.778924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811c0df-a13d-4c0a-9514-97d128a498c4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:32.783902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0bbbd-faab-4e12-9611-540144ef2f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:32.784556 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0bbbd-faab-4e12-9611-540144ef2f36 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:32.784846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0bbbd-faab-4e12-9611-540144ef2f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:32.785125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0bbbd-faab-4e12-9611-540144ef2f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:32.795686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0bbbd-faab-4e12-9611-540144ef2f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:32.797190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0bbbd-faab-4e12-9611-540144ef2f36 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:32.815847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1362/2728] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:33.234146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16a59a-fb46-4b09-b630-c5c92d14ec38 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:33.236995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae16a59a-fb46-4b09-b630-c5c92d14ec38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:33.270381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16a59a-fb46-4b09-b630-c5c92d14ec38 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:33.279250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:33.281076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:33.281597 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:33.283025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] PATH_INFO: `/v3/projects/cded828d48b042e99e33885391c0c3e4` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:33.285718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cded828d48b042e99e33885391c0c3e4)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:33.290629 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:33.378362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cded828d48b042e99e33885391c0c3e4', 'request_id': 'req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:33.389766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cded828d48b042e99e33885391c0c3e4', 'request_id': 'req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:33.403242 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cded828d48b042e99e33885391c0c3e4', 'request_id': 'req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:33.428157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] The token cache is being invalidate because project cded828d48b042e99e33885391c0c3e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:33.428690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:33.429766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5b37b265-fbbc-48f2-8b5f-8a3fbe34645d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:33.432116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1367/2729] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:33 2022] DELETE /identity/v3/projects/cded828d48b042e99e33885391c0c3e4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:33.439731 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b658c-45aa-4d3a-8c5c-0b0cfa86f966 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:33.440384 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b658c-45aa-4d3a-8c5c-0b0cfa86f966 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:33.440813 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b658c-45aa-4d3a-8c5c-0b0cfa86f966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:33.478604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-f58b658c-45aa-4d3a-8c5c-0b0cfa86f966 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:33.480095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f58b658c-45aa-4d3a-8c5c-0b0cfa86f966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:33.573198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1363/2730] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:33.583458 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41d03-1752-4ef0-8ebd-02d24f4fb3ec None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:33.584016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41d03-1752-4ef0-8ebd-02d24f4fb3ec None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:33.584016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b41d03-1752-4ef0-8ebd-02d24f4fb3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:33.616044 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b5b41d03-1752-4ef0-8ebd-02d24f4fb3ec None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:33.618167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b5b41d03-1752-4ef0-8ebd-02d24f4fb3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:33.636216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1368/2731] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:33.638019 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b658c-45aa-4d3a-8c5c-0b0cfa86f966 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:33.650953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b658c-45aa-4d3a-8c5c-0b0cfa86f966 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:33.657023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:33.658424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:33.658770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:33.659143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:33.659890 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:40:33.662097 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:33.663836 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:33.680422 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1364/2732] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:40:33.689168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b41d03-1752-4ef0-8ebd-02d24f4fb3ec None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:33.707480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b41d03-1752-4ef0-8ebd-02d24f4fb3ec None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:33.716330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb01dda-2612-46bb-b471-308f1fc96b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:33.717378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb01dda-2612-46bb-b471-308f1fc96b86 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:33.717925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb01dda-2612-46bb-b471-308f1fc96b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:33.718417 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb01dda-2612-46bb-b471-308f1fc96b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:33.719561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb01dda-2612-46bb-b471-308f1fc96b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:33.723697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb01dda-2612-46bb-b471-308f1fc96b86 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:33.789852 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1369/2733] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:40:33 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:33.795960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:33.815783 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac523b-4192-4fcd-bd4f-d3509701966f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:33.868165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:33.906639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fde71-056d-461f-9919-f7501570f3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:33.908087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fde71-056d-461f-9919-f7501570f3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:33.908300 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fde71-056d-461f-9919-f7501570f3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:33.908636 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fde71-056d-461f-9919-f7501570f3f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:33.909318 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fde71-056d-461f-9919-f7501570f3f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:33.911404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fde71-056d-461f-9919-f7501570f3f5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:33.935767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1365/2734] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:40:33 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:40:33.940512 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb01dda-2612-46bb-b471-308f1fc96b86 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:33.947448 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:33.956338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb01dda-2612-46bb-b471-308f1fc96b86 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:33.965040 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6be67-8707-4a63-8a6d-d7684394e2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:33.966560 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6be67-8707-4a63-8a6d-d7684394e2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:33.967273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6be67-8707-4a63-8a6d-d7684394e2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:33.968011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6be67-8707-4a63-8a6d-d7684394e2e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:33.969155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6be67-8707-4a63-8a6d-d7684394e2e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:33.972809 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6be67-8707-4a63-8a6d-d7684394e2e2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:33.990989 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1370/2735] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:33.995253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fde71-056d-461f-9919-f7501570f3f5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:34.008616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fde71-056d-461f-9919-f7501570f3f5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:34.015081 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c47be0-0ef8-4b17-820d-57ad1d7df1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:34.015818 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c47be0-0ef8-4b17-820d-57ad1d7df1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:34.016177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c47be0-0ef8-4b17-820d-57ad1d7df1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:34.016456 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c47be0-0ef8-4b17-820d-57ad1d7df1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:34.017876 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c47be0-0ef8-4b17-820d-57ad1d7df1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:34.019792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c47be0-0ef8-4b17-820d-57ad1d7df1b8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:34.031886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1366/2736] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:40:34.037926 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6be67-8707-4a63-8a6d-d7684394e2e2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:34.053134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6be67-8707-4a63-8a6d-d7684394e2e2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:34.058764 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:34.059575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:34.059970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:34.060346 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] PATH_INFO: `/v3/projects/8c9a6a9d739f48b6957572c7bf691ed1/users/098db53b110c4075b1814d74e78f9969/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:34.081611 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9a6a9d739f48b6957572c7bf691ed1, user_id=098db53b110c4075b1814d74e78f9969, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:34.084672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:34.102864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1371/2737] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:40:34 2022] PUT /identity/v3/projects/8c9a6a9d739f48b6957572c7bf691ed1/users/098db53b110c4075b1814d74e78f9969/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:40:34.117000 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c77e-4f27-475c-b7f2-0254648a2bce None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:34.117540 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c77e-4f27-475c-b7f2-0254648a2bce None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:34.117903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c77e-4f27-475c-b7f2-0254648a2bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:34.161483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2ba6c77e-4f27-475c-b7f2-0254648a2bce None None] MFA Rules not processed for user `098db53b110c4075b1814d74e78f9969`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:34.162863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2ba6c77e-4f27-475c-b7f2-0254648a2bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:34.299664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1367/2738] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:34.311209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:34.313265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:34.411598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb277693-5d99-4387-9fd6-a5a76cc40a6a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:34.416356 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d18f4-15ef-4dbc-a510-ae730d26e01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:34.417148 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d18f4-15ef-4dbc-a510-ae730d26e01d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:34.417533 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d18f4-15ef-4dbc-a510-ae730d26e01d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:34.417942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d18f4-15ef-4dbc-a510-ae730d26e01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:34.432992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d18f4-15ef-4dbc-a510-ae730d26e01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:34.434917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d18f4-15ef-4dbc-a510-ae730d26e01d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:34.449101 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1372/2739] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:44.295321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:44.442884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:44.652403 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6c77e-4f27-475c-b7f2-0254648a2bce None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:44.654589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2ba6c77e-4f27-475c-b7f2-0254648a2bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:44.668312 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6c77e-4f27-475c-b7f2-0254648a2bce None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:44.672352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb8240-cb79-4815-93f1-d08c63651c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:44.673233 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb8240-cb79-4815-93f1-d08c63651c96 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:44.673627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb8240-cb79-4815-93f1-d08c63651c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:44.673905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb8240-cb79-4815-93f1-d08c63651c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:44.675798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a6fb8240-cb79-4815-93f1-d08c63651c96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:44.774513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb8240-cb79-4815-93f1-d08c63651c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:44.776258 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb8240-cb79-4815-93f1-d08c63651c96 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:44.797412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1368/2740] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:45.552914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d18f4-15ef-4dbc-a510-ae730d26e01d service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:45.625530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d18f4-15ef-4dbc-a510-ae730d26e01d service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:45.631115 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:45.631766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:45.632049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:45.632327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] PATH_INFO: `/v3/users/098db53b110c4075b1814d74e78f9969` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:45.633501 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=098db53b110c4075b1814d74e78f9969)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:45.635465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:45.686699 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '098db53b110c4075b1814d74e78f9969'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:45.699612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '098db53b110c4075b1814d74e78f9969'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:45.700388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '098db53b110c4075b1814d74e78f9969'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:45.709797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '098db53b110c4075b1814d74e78f9969'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:45.717751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '098db53b110c4075b1814d74e78f9969'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:45.729124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1373/2741] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:40:45 2022] DELETE /identity/v3/users/098db53b110c4075b1814d74e78f9969 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:45.734014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849e16-fc22-4d84-bfaf-fee04ff1a635 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:45.734574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849e16-fc22-4d84-bfaf-fee04ff1a635 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:45.734574 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14849e16-fc22-4d84-bfaf-fee04ff1a635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:45.765883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-14849e16-fc22-4d84-bfaf-fee04ff1a635 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:45.768562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-14849e16-fc22-4d84-bfaf-fee04ff1a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:45.873693 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1369/2742] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:45.885174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:45.886750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:45.954058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd4ba1-ad7a-4607-952a-817ff2c40a0d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:45.957922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:45.958556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:45.958840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:45.959122 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:45.969023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:45.970661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:45.980605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1374/2743] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:46.420280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14849e16-fc22-4d84-bfaf-fee04ff1a635 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:46.423678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-14849e16-fc22-4d84-bfaf-fee04ff1a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:46.449734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14849e16-fc22-4d84-bfaf-fee04ff1a635 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:46.462871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:46.463906 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:46.464499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:46.465308 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] PATH_INFO: `/v3/projects/8c9a6a9d739f48b6957572c7bf691ed1` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:46.468404 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9a6a9d739f48b6957572c7bf691ed1)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:46.473029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:46.573659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9a6a9d739f48b6957572c7bf691ed1', 'request_id': 'req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:46.582553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9a6a9d739f48b6957572c7bf691ed1', 'request_id': 'req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:46.589769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9a6a9d739f48b6957572c7bf691ed1', 'request_id': 'req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:46.613161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] The token cache is being invalidate because project 8c9a6a9d739f48b6957572c7bf691ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:40:46.613896 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:46.615174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-8fafd9bc-ed96-4914-ac3a-e0ddc65102a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:40:46.618028 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1370/2744] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:40:46 2022] DELETE /identity/v3/projects/8c9a6a9d739f48b6957572c7bf691ed1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:40:47.849966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:40:47.851515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:47.864610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b752424-5a8c-43ab-943f-e6d686d1ed71 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:40:47.887049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:47.932237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cc681-3d3f-40af-851c-d6df3eade269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:40:47.932853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cc681-3d3f-40af-851c-d6df3eade269 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:47.933133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cc681-3d3f-40af-851c-d6df3eade269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:47.933408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cc681-3d3f-40af-851c-d6df3eade269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:47.935179 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-089cc681-3d3f-40af-851c-d6df3eade269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:48.017523 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cc681-3d3f-40af-851c-d6df3eade269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:40:48.018934 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cc681-3d3f-40af-851c-d6df3eade269 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:40:48.031660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1375/2745] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:40:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:40:49.487579 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c1618-18d3-490b-a36a-c397ae00cc17 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:49.488133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c1618-18d3-490b-a36a-c397ae00cc17 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:49.488133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c1618-18d3-490b-a36a-c397ae00cc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:49.510425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-949c1618-18d3-490b-a36a-c397ae00cc17 None None] MFA Rules not processed for user `f548ccf8c5a14b108aa8b851560139d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:49.511903 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-949c1618-18d3-490b-a36a-c397ae00cc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:49.517668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:40:49.592525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1371/2746] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:40:53.859411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913dcd7-2e7c-4c0b-b664-95491cab91fa None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:40:53.860623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913dcd7-2e7c-4c0b-b664-95491cab91fa None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:40:53.860623 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913dcd7-2e7c-4c0b-b664-95491cab91fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:40:53.885706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c913dcd7-2e7c-4c0b-b664-95491cab91fa None None] MFA Rules not processed for user `21fde7fa2a16426890bdcba2241778ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:40:53.886958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c913dcd7-2e7c-4c0b-b664-95491cab91fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:40:53.949029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1376/2747] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:40:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:40:59.584801 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=21 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:41:03.942057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=69 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:41:10.235408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c1618-18d3-490b-a36a-c397ae00cc17 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:41:10.237099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-949c1618-18d3-490b-a36a-c397ae00cc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:41:10.249831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c1618-18d3-490b-a36a-c397ae00cc17 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:41:10.254260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748a765-f717-47c2-812f-13964254228b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:41:10.254866 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a765-f717-47c2-812f-13964254228b service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:41:10.255148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a765-f717-47c2-812f-13964254228b service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:41:10.255439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748a765-f717-47c2-812f-13964254228b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:41:10.257516 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2748a765-f717-47c2-812f-13964254228b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:41:10.268762 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748a765-f717-47c2-812f-13964254228b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:41:10.270390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748a765-f717-47c2-812f-13964254228b service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:41:10.279120 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1372/2748] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:41:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:41:12.542195 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913dcd7-2e7c-4c0b-b664-95491cab91fa None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:41:12.553721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913dcd7-2e7c-4c0b-b664-95491cab91fa None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:41:12.557734 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:41:12.558363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:41:12.558661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:41:12.558935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:41:12.560779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:41:12.632175 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:41:12.633685 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:41:12.645266 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1377/2749] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:41:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:41:20.275108 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:41:22.639564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=30 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:42:20.132673 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492fa80-3e63-402c-8052-4f5354820422 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:20.133437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492fa80-3e63-402c-8052-4f5354820422 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:20.133437 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492fa80-3e63-402c-8052-4f5354820422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:20.201209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-4492fa80-3e63-402c-8052-4f5354820422 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:42:20.202495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4492fa80-3e63-402c-8052-4f5354820422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:20.416840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1373/2750] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:42:20.474259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:20.475853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:20.581260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868caf1a-895a-48cc-b784-348f8c8811a1 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:20.587411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64c885-ad67-4a47-b23c-ffddad74e9a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:20.588039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c885-ad67-4a47-b23c-ffddad74e9a8 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:20.588327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c885-ad67-4a47-b23c-ffddad74e9a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:20.588607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64c885-ad67-4a47-b23c-ffddad74e9a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:20.598805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64c885-ad67-4a47-b23c-ffddad74e9a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:20.600587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64c885-ad67-4a47-b23c-ffddad74e9a8 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:20.609575 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1378/2751] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:42:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:42:30.413654 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=40 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:42:30.605834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=41 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:42:35.346924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25463d2-bba9-4fb1-bf16-c0a056c7abaf None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.348069 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25463d2-bba9-4fb1-bf16-c0a056c7abaf None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.348494 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25463d2-bba9-4fb1-bf16-c0a056c7abaf None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.350410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1374/2752] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:42:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:42:35.357763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f98c-6256-4b42-b00e-1057cf7c54f7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.358325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f98c-6256-4b42-b00e-1057cf7c54f7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.358539 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f98c-6256-4b42-b00e-1057cf7c54f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.411013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4a29f98c-6256-4b42-b00e-1057cf7c54f7 None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:42:35.412410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4a29f98c-6256-4b42-b00e-1057cf7c54f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:35.422599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1379/2753] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:42:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:42:35.436445 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25463d2-bba9-4fb1-bf16-c0a056c7abaf None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:35.438155 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c25463d2-bba9-4fb1-bf16-c0a056c7abaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:35.530092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25463d2-bba9-4fb1-bf16-c0a056c7abaf None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:35.533814 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce10d8-e054-46e2-bcea-64a60f37f512 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '464190e0ccaf4f82942f518df1bf6c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:35.534435 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce10d8-e054-46e2-bcea-64a60f37f512 service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.534715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce10d8-e054-46e2-bcea-64a60f37f512 service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.534988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce10d8-e054-46e2-bcea-64a60f37f512 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.546065 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce10d8-e054-46e2-bcea-64a60f37f512 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:35.547507 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce10d8-e054-46e2-bcea-64a60f37f512 service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:35.559710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1375/2754] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:42:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:42:35.632154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1e7a-8ca8-4e5d-8b80-3602ef67800b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.632445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1e7a-8ca8-4e5d-8b80-3602ef67800b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.632746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a1e7a-8ca8-4e5d-8b80-3602ef67800b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.634124 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1380/2755] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:42:35.639635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b7682-8517-46a5-a14b-4cc48d6597ca None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.639959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b7682-8517-46a5-a14b-4cc48d6597ca None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.640249 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b7682-8517-46a5-a14b-4cc48d6597ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.680879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8b5b7682-8517-46a5-a14b-4cc48d6597ca None None] MFA Rules not processed for user `cf865cb3c7fc432884e4e308c397c83d`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:42:35.682148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8b5b7682-8517-46a5-a14b-4cc48d6597ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:35.688892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1376/2756] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:42:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:42:35.701963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1e7a-8ca8-4e5d-8b80-3602ef67800b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:35.715759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a1e7a-8ca8-4e5d-8b80-3602ef67800b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:35.720084 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d4f60-d19d-4b02-867f-47d7028f7b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:35.720697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4f60-d19d-4b02-867f-47d7028f7b42 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.720976 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4f60-d19d-4b02-867f-47d7028f7b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.721253 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d4f60-d19d-4b02-867f-47d7028f7b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.729402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d4f60-d19d-4b02-867f-47d7028f7b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:35.731104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d4f60-d19d-4b02-867f-47d7028f7b42 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:35.741205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1381/2757] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:42:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:42:35.828208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154facc-cc9c-452d-ad3c-3b1eb2c4a13f None None] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.828562 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154facc-cc9c-452d-ad3c-3b1eb2c4a13f None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.828840 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154facc-cc9c-452d-ad3c-3b1eb2c4a13f None None] PATH_INFO: `/` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.830264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1377/2758] 10.222.0.58 () {54 vars in 921 bytes} [Mon Oct 3 13:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 03 13:42:35.836651 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5a14-cbdd-4f48-87c6-438db799599b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.836935 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5a14-cbdd-4f48-87c6-438db799599b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.837270 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee5a14-cbdd-4f48-87c6-438db799599b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.869080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-17ee5a14-cbdd-4f48-87c6-438db799599b None None] MFA Rules not processed for user `c3ece3405a2b43288136ddb79b84a106`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:42:35.870355 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-17ee5a14-cbdd-4f48-87c6-438db799599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:35.880517 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1382/2759] 10.222.0.58 () {58 vars in 1018 bytes} [Mon Oct 3 13:42:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:42:35.893113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154facc-cc9c-452d-ad3c-3b1eb2c4a13f None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:35.895763 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-6154facc-cc9c-452d-ad3c-3b1eb2c4a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:35.910487 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154facc-cc9c-452d-ad3c-3b1eb2c4a13f None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:35.914929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbca046-b7ab-4301-a9ca-9d199e879d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:35.915585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbca046-b7ab-4301-a9ca-9d199e879d9f service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:35.915897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbca046-b7ab-4301-a9ca-9d199e879d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:35.916176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbca046-b7ab-4301-a9ca-9d199e879d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:35.928177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbca046-b7ab-4301-a9ca-9d199e879d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:35.929974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbca046-b7ab-4301-a9ca-9d199e879d9f service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:35.941721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1378/2760] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:42:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:42:36.017247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee5a14-cbdd-4f48-87c6-438db799599b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:36.018706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-17ee5a14-cbdd-4f48-87c6-438db799599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:36.104025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee5a14-cbdd-4f48-87c6-438db799599b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:36.109639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:36.110276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:36.110565 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:36.110846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:36.112706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:36.126207 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:36.127846 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:36.140113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1383/2761] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:42:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:42:38.013111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627065a-47bb-47cf-ae52-a98be14e0437 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:38.013863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627065a-47bb-47cf-ae52-a98be14e0437 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:38.013863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627065a-47bb-47cf-ae52-a98be14e0437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:38.036164 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8627065a-47bb-47cf-ae52-a98be14e0437 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:42:38.037873 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8627065a-47bb-47cf-ae52-a98be14e0437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:38.045594 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1379/2762] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:42:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:42:38.061767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:38.074390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba61851-491b-447c-be2e-7c508b7d76b1 service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:38.080097 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcb0ab-8450-4204-8302-d920a473ca01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:38.080914 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcb0ab-8450-4204-8302-d920a473ca01 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:38.081230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcb0ab-8450-4204-8302-d920a473ca01 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:38.081506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcb0ab-8450-4204-8302-d920a473ca01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:38.092081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfcb0ab-8450-4204-8302-d920a473ca01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:38.093827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfcb0ab-8450-4204-8302-d920a473ca01 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:38.104556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1384/2763] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:42:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:42:48.029293 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=53 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:42:48.100450 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=56 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:42:50.163145 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627065a-47bb-47cf-ae52-a98be14e0437 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:50.164551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8627065a-47bb-47cf-ae52-a98be14e0437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:50.180736 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627065a-47bb-47cf-ae52-a98be14e0437 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:50.186244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3211b-f402-480d-b73a-418e35329d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:50.186859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3211b-f402-480d-b73a-418e35329d1c service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:50.187140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3211b-f402-480d-b73a-418e35329d1c service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:50.187410 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3211b-f402-480d-b73a-418e35329d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:50.189173 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-4cb3211b-f402-480d-b73a-418e35329d1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:50.202027 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3211b-f402-480d-b73a-418e35329d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:50.203724 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3211b-f402-480d-b73a-418e35329d1c service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:50.214531 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1380/2764] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:42:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:42:52.376755 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcb0ab-8450-4204-8302-d920a473ca01 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:52.390789 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcb0ab-8450-4204-8302-d920a473ca01 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:52.397116 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:52.397780 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:52.398059 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:52.398330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:52.400164 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:52.415878 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:52.417571 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:52.429902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1385/2765] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:42:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:42:54.094638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81c60-2219-4e90-a51c-1e245456fd91 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:54.095159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81c60-2219-4e90-a51c-1e245456fd91 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:54.095282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b81c60-2219-4e90-a51c-1e245456fd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:54.127555 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-27b81c60-2219-4e90-a51c-1e245456fd91 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:42:54.129335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-27b81c60-2219-4e90-a51c-1e245456fd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:42:54.136761 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1381/2766] 10.222.0.58 () {58 vars in 967 bytes} [Mon Oct 3 13:42:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:42:54.152139 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:42:54.168659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e668a-e236-43de-a8f7-1ed834c182e9 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:42:54.174070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f21047-41c0-40ff-9c98-e0abe30209b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:42:54.174697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f21047-41c0-40ff-9c98-e0abe30209b9 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:42:54.174979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f21047-41c0-40ff-9c98-e0abe30209b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:42:54.175252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f21047-41c0-40ff-9c98-e0abe30209b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:42:54.187138 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f21047-41c0-40ff-9c98-e0abe30209b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:42:54.189888 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f21047-41c0-40ff-9c98-e0abe30209b9 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:42:54.207459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1386/2767] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:42:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:43:04.121819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=9 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:04.200605 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=12 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:26.313063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b81c60-2219-4e90-a51c-1e245456fd91 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:26.328775 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b81c60-2219-4e90-a51c-1e245456fd91 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:26.339710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:26.340478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:26.340927 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:26.341331 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:26.343639 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:26.357957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:26.359688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:26.387700 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1382/2768] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:43:28.003759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0172db-7958-4676-856e-a1b582afe11d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:28.004234 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0172db-7958-4676-856e-a1b582afe11d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:28.004336 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0172db-7958-4676-856e-a1b582afe11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:28.024219 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cd0172db-7958-4676-856e-a1b582afe11d None None] MFA Rules not processed for user `f548ccf8c5a14b108aa8b851560139d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:28.025441 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cd0172db-7958-4676-856e-a1b582afe11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:28.032338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1387/2769] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:36.372222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=6 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:38.019330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:40.521587 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:40.534103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960f32-86ec-411d-83b8-6565f75e4a30 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:40.538930 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:40.540664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:40.540850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:40.541526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:40.543649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:40.631628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:40.635339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:40.646877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1383/2770] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:43:44.484379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0172db-7958-4676-856e-a1b582afe11d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:44.491098 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cd0172db-7958-4676-856e-a1b582afe11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:44.507956 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0172db-7958-4676-856e-a1b582afe11d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:44.513376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:44.516152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:44.516152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:44.516152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] PATH_INFO: `/v3/users/21fde7fa2a16426890bdcba2241778ed` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:44.516152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21fde7fa2a16426890bdcba2241778ed)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:44.517690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:44.570264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21fde7fa2a16426890bdcba2241778ed'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.583196 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21fde7fa2a16426890bdcba2241778ed'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.583902 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21fde7fa2a16426890bdcba2241778ed'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.593624 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21fde7fa2a16426890bdcba2241778ed'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.602481 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-733a07b3-62ce-46aa-9a99-a165ddfae9a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21fde7fa2a16426890bdcba2241778ed'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.608413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1388/2771] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:43:44 2022] DELETE /identity/v3/users/21fde7fa2a16426890bdcba2241778ed => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:44.798656 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:44.811071 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342377b-bcc2-4320-ab09-e5b7a5932200 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:44.874285 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:44.875193 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:44.875473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:44.875747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] PATH_INFO: `/v3/projects/e019f687769f408d9c3faab0c8693a9d` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:44.876877 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e019f687769f408d9c3faab0c8693a9d)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:44.878915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:44.937889 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e019f687769f408d9c3faab0c8693a9d', 'request_id': 'req-135de3d6-ae21-41f1-8ff8-956f6c225d64'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.946568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e019f687769f408d9c3faab0c8693a9d', 'request_id': 'req-135de3d6-ae21-41f1-8ff8-956f6c225d64'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.955941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e019f687769f408d9c3faab0c8693a9d', 'request_id': 'req-135de3d6-ae21-41f1-8ff8-956f6c225d64'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.972534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] The token cache is being invalidate because project e019f687769f408d9c3faab0c8693a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:43:44.972756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.973558 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-135de3d6-ae21-41f1-8ff8-956f6c225d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:44.975152 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1384/2772] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:43:44 2022] DELETE /identity/v3/projects/e019f687769f408d9c3faab0c8693a9d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:43:44.980979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65431-a74e-4cd6-aec8-9265c22571de None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:44.981380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65431-a74e-4cd6-aec8-9265c22571de None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:44.981794 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65431-a74e-4cd6-aec8-9265c22571de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.004631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9db65431-a74e-4cd6-aec8-9265c22571de None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:45.006310 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9db65431-a74e-4cd6-aec8-9265c22571de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:45.079703 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1389/2773] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:45.089198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a2b5-5099-43f6-924d-8eab0db023f3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.089519 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a2b5-5099-43f6-924d-8eab0db023f3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.089805 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64a2b5-5099-43f6-924d-8eab0db023f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.111622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ad64a2b5-5099-43f6-924d-8eab0db023f3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:45.112883 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ad64a2b5-5099-43f6-924d-8eab0db023f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:45.121076 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1385/2774] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:45.125695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db65431-a74e-4cd6-aec8-9265c22571de None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.135869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db65431-a74e-4cd6-aec8-9265c22571de None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.140775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.141392 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.141706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.141988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.142657 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:43:45.142963 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.144585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.157640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1390/2775] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:43:45.162131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64a2b5-5099-43f6-924d-8eab0db023f3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.172045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64a2b5-5099-43f6-924d-8eab0db023f3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.176739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becee13b-0d13-49c3-b633-279b264fe8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.177349 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becee13b-0d13-49c3-b633-279b264fe8aa admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.177732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becee13b-0d13-49c3-b633-279b264fe8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.178003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becee13b-0d13-49c3-b633-279b264fe8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.178650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becee13b-0d13-49c3-b633-279b264fe8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.180589 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becee13b-0d13-49c3-b633-279b264fe8aa admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.197282 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:45.214620 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1386/2776] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:43:45 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:43:45.219007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.229859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d080cd8-984f-4d1f-829b-db8a15681c6d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.293180 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4866-100c-4f2a-894a-2a527721e762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.293853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4866-100c-4f2a-894a-2a527721e762 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.294136 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4866-100c-4f2a-894a-2a527721e762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.294413 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4866-100c-4f2a-894a-2a527721e762 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.295060 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4866-100c-4f2a-894a-2a527721e762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.297158 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4866-100c-4f2a-894a-2a527721e762 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.305697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:45.320546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1391/2777] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:43:45 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:43:45.324800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becee13b-0d13-49c3-b633-279b264fe8aa admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.335631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becee13b-0d13-49c3-b633-279b264fe8aa admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.340962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be60ff2-5c66-474e-9f68-c22cbcfdcb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.341628 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be60ff2-5c66-474e-9f68-c22cbcfdcb00 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.341932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be60ff2-5c66-474e-9f68-c22cbcfdcb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.342207 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be60ff2-5c66-474e-9f68-c22cbcfdcb00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.342855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be60ff2-5c66-474e-9f68-c22cbcfdcb00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.344466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be60ff2-5c66-474e-9f68-c22cbcfdcb00 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.356939 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1387/2778] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:45.361193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4866-100c-4f2a-894a-2a527721e762 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.371853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4866-100c-4f2a-894a-2a527721e762 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.377061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e37d5e-3c9b-4e27-9709-775569de1c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.377701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e37d5e-3c9b-4e27-9709-775569de1c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.377992 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e37d5e-3c9b-4e27-9709-775569de1c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.378275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e37d5e-3c9b-4e27-9709-775569de1c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.379065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e37d5e-3c9b-4e27-9709-775569de1c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.380706 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e37d5e-3c9b-4e27-9709-775569de1c10 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.392904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1392/2779] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:43:45.397015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be60ff2-5c66-474e-9f68-c22cbcfdcb00 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.409034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be60ff2-5c66-474e-9f68-c22cbcfdcb00 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.414929 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a77c5-87e3-47ad-bf07-90d6073c3eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.416018 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a77c5-87e3-47ad-bf07-90d6073c3eba admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.416295 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a77c5-87e3-47ad-bf07-90d6073c3eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.416569 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a77c5-87e3-47ad-bf07-90d6073c3eba admin admin] PATH_INFO: `/v3/projects/fee4e82230074e3985884e361b46adaf/users/9bb24412ab4a455b859faa8b944d371f/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.432626 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a77c5-87e3-47ad-bf07-90d6073c3eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee4e82230074e3985884e361b46adaf, user_id=9bb24412ab4a455b859faa8b944d371f, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.435309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a77c5-87e3-47ad-bf07-90d6073c3eba admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.451107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1388/2780] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:43:45 2022] PUT /identity/v3/projects/fee4e82230074e3985884e361b46adaf/users/9bb24412ab4a455b859faa8b944d371f/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:45.462807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02141baa-4f63-4485-9e3f-35cf04bd0ea5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.463092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02141baa-4f63-4485-9e3f-35cf04bd0ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.463364 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02141baa-4f63-4485-9e3f-35cf04bd0ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.498540 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-02141baa-4f63-4485-9e3f-35cf04bd0ea5 None None] MFA Rules not processed for user `9bb24412ab4a455b859faa8b944d371f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:45.500030 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-02141baa-4f63-4485-9e3f-35cf04bd0ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:45.591065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1393/2781] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:45.596471 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a77c5-87e3-47ad-bf07-90d6073c3eba admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.669728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a77c5-87e3-47ad-bf07-90d6073c3eba admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.674429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bba37-0fa4-4bc8-bc71-54552fab1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.675134 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bba37-0fa4-4bc8-bc71-54552fab1fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.675325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bba37-0fa4-4bc8-bc71-54552fab1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.675599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bba37-0fa4-4bc8-bc71-54552fab1fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.676275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bba37-0fa4-4bc8-bc71-54552fab1fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.678369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bba37-0fa4-4bc8-bc71-54552fab1fd4 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.706853 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1389/2782] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:43:45 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:43:45.712054 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02141baa-4f63-4485-9e3f-35cf04bd0ea5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.723941 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02141baa-4f63-4485-9e3f-35cf04bd0ea5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.785178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c269f-a70d-468f-ab6e-b201b7ee5f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.785746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c269f-a70d-468f-ab6e-b201b7ee5f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.786037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c269f-a70d-468f-ab6e-b201b7ee5f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.786329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c269f-a70d-468f-ab6e-b201b7ee5f31 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.786986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c269f-a70d-468f-ab6e-b201b7ee5f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.788939 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c269f-a70d-468f-ab6e-b201b7ee5f31 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.813970 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1394/2783] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:43:45 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:43:45.818605 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bba37-0fa4-4bc8-bc71-54552fab1fd4 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.831534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bba37-0fa4-4bc8-bc71-54552fab1fd4 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.836696 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436f76-a7dd-44e4-acb1-5a7298e3aa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.837321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436f76-a7dd-44e4-acb1-5a7298e3aa76 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.837638 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436f76-a7dd-44e4-acb1-5a7298e3aa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.837915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436f76-a7dd-44e4-acb1-5a7298e3aa76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.838575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e436f76-a7dd-44e4-acb1-5a7298e3aa76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.840200 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e436f76-a7dd-44e4-acb1-5a7298e3aa76 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.852158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1390/2784] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:45.856962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c269f-a70d-468f-ab6e-b201b7ee5f31 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.870265 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c269f-a70d-468f-ab6e-b201b7ee5f31 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.876416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaea3b2-d21a-4370-9d7d-9febce395426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.877291 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaea3b2-d21a-4370-9d7d-9febce395426 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.877786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaea3b2-d21a-4370-9d7d-9febce395426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.878194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaea3b2-d21a-4370-9d7d-9febce395426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.879149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaea3b2-d21a-4370-9d7d-9febce395426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.881704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaea3b2-d21a-4370-9d7d-9febce395426 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.896250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1395/2785] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:45.901800 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436f76-a7dd-44e4-acb1-5a7298e3aa76 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:45.912918 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436f76-a7dd-44e4-acb1-5a7298e3aa76 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:45.919127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2762ea-4d25-4fb0-9775-5c64ae4d940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:45.919815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2762ea-4d25-4fb0-9775-5c64ae4d940d admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.920079 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2762ea-4d25-4fb0-9775-5c64ae4d940d admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.920350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2762ea-4d25-4fb0-9775-5c64ae4d940d admin admin] PATH_INFO: `/v3/projects/72cc56f86f3e4f2bbb9e71eac3896efb/users/96c36d55e61e4a278adef15e355c799c/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.937892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2762ea-4d25-4fb0-9775-5c64ae4d940d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72cc56f86f3e4f2bbb9e71eac3896efb, user_id=96c36d55e61e4a278adef15e355c799c, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:45.941679 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2762ea-4d25-4fb0-9775-5c64ae4d940d admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:45.957960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1391/2786] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:43:45 2022] PUT /identity/v3/projects/72cc56f86f3e4f2bbb9e71eac3896efb/users/96c36d55e61e4a278adef15e355c799c/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:45.968739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19bcce-0431-439a-8ee8-c839e981f42a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:45.969067 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19bcce-0431-439a-8ee8-c839e981f42a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:45.969275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19bcce-0431-439a-8ee8-c839e981f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:45.999273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-de19bcce-0431-439a-8ee8-c839e981f42a None None] MFA Rules not processed for user `96c36d55e61e4a278adef15e355c799c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:46.000832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-de19bcce-0431-439a-8ee8-c839e981f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:46.039003 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:46.074874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1396/2787] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:43:46.080479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2762ea-4d25-4fb0-9775-5c64ae4d940d admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:46.162154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2762ea-4d25-4fb0-9775-5c64ae4d940d admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:46.170951 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b810af-292c-4f68-b933-d6b6abc28424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:46.171988 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b810af-292c-4f68-b933-d6b6abc28424 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.172451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b810af-292c-4f68-b933-d6b6abc28424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.172902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b810af-292c-4f68-b933-d6b6abc28424 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:46.174034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b810af-292c-4f68-b933-d6b6abc28424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:46.176434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b810af-292c-4f68-b933-d6b6abc28424 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:46.196077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:46.210719 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1392/2788] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:43:46 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:43:46.214994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19bcce-0431-439a-8ee8-c839e981f42a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:46.229047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19bcce-0431-439a-8ee8-c839e981f42a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:46.297959 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049de0e6-eb98-45e0-8b45-aa9c2267e02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:46.298596 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049de0e6-eb98-45e0-8b45-aa9c2267e02d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.298884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049de0e6-eb98-45e0-8b45-aa9c2267e02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.299167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049de0e6-eb98-45e0-8b45-aa9c2267e02d admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:46.299816 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049de0e6-eb98-45e0-8b45-aa9c2267e02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:46.301790 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049de0e6-eb98-45e0-8b45-aa9c2267e02d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:46.330466 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1397/2789] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:43:46 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:43:46.337206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b810af-292c-4f68-b933-d6b6abc28424 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:46.350682 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b810af-292c-4f68-b933-d6b6abc28424 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:46.360137 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c291ae7-3e03-4df8-8020-932f3daeb03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:46.360684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c291ae7-3e03-4df8-8020-932f3daeb03f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.360967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c291ae7-3e03-4df8-8020-932f3daeb03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.361546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c291ae7-3e03-4df8-8020-932f3daeb03f admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:46.362183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c291ae7-3e03-4df8-8020-932f3daeb03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:46.364023 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c291ae7-3e03-4df8-8020-932f3daeb03f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:46.376609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1393/2790] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:46.381187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049de0e6-eb98-45e0-8b45-aa9c2267e02d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:46.392670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049de0e6-eb98-45e0-8b45-aa9c2267e02d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:46.398733 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b937b4-40a4-4777-a3bb-6c4e118769cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:46.400108 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b937b4-40a4-4777-a3bb-6c4e118769cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.400391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b937b4-40a4-4777-a3bb-6c4e118769cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.400672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b937b4-40a4-4777-a3bb-6c4e118769cc admin admin] PATH_INFO: `/v3/projects/854fbd3261074ed786c825fcb8a1a64e/users/b1cff275e6154a1c9bf5229ccdd28354/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:46.418092 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b937b4-40a4-4777-a3bb-6c4e118769cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=854fbd3261074ed786c825fcb8a1a64e, user_id=b1cff275e6154a1c9bf5229ccdd28354, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:46.420895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b937b4-40a4-4777-a3bb-6c4e118769cc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:46.439488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1398/2791] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:43:46 2022] PUT /identity/v3/projects/854fbd3261074ed786c825fcb8a1a64e/users/b1cff275e6154a1c9bf5229ccdd28354/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:46.444268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c291ae7-3e03-4df8-8020-932f3daeb03f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:46.462401 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c291ae7-3e03-4df8-8020-932f3daeb03f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:46.550932 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cac64-aa08-4d7b-988b-fece011d1f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:46.551480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cac64-aa08-4d7b-988b-fece011d1f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.551790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cac64-aa08-4d7b-988b-fece011d1f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.552059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cac64-aa08-4d7b-988b-fece011d1f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:46.553221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cac64-aa08-4d7b-988b-fece011d1f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:46.554378 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cac64-aa08-4d7b-988b-fece011d1f50 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:46.569998 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1394/2792] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:46.573039 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b937b4-40a4-4777-a3bb-6c4e118769cc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:46.592252 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b937b4-40a4-4777-a3bb-6c4e118769cc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:46.600642 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:46.602445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.602940 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.603459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] PATH_INFO: `/v3/projects/854fbd3261074ed786c825fcb8a1a64e/users/b1cff275e6154a1c9bf5229ccdd28354/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:46.606225 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854fbd3261074ed786c825fcb8a1a64e, user_id=b1cff275e6154a1c9bf5229ccdd28354, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:46.610549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:46.623866 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1399/2793] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:43:46 2022] PUT /identity/v3/projects/854fbd3261074ed786c825fcb8a1a64e/users/b1cff275e6154a1c9bf5229ccdd28354/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:46.638823 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00008-4cf9-49cb-9025-deb0e3b2157c None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.639296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00008-4cf9-49cb-9025-deb0e3b2157c None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.639718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be00008-4cf9-49cb-9025-deb0e3b2157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:46.680722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-3be00008-4cf9-49cb-9025-deb0e3b2157c None None] MFA Rules not processed for user `b1cff275e6154a1c9bf5229ccdd28354`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:46.682845 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3be00008-4cf9-49cb-9025-deb0e3b2157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:46.774714 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1395/2794] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:43:46.786658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:46.788589 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:46.864791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff1062-820c-45af-8a0c-c3e5e8097403 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:46.868640 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:46.869312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.869636 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.869912 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:46.880399 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:46.882701 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:46.894673 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1400/2795] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:43:46.944787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be00008-4cf9-49cb-9025-deb0e3b2157c None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:46.947970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3be00008-4cf9-49cb-9025-deb0e3b2157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:46.962970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be00008-4cf9-49cb-9025-deb0e3b2157c None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:46.968289 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:46.968967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:46.969280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:46.969596 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:47.039615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:47.041125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:47.050777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1396/2796] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:43:48.238206 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:48.239634 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:48.278107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:48.322237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c31a5-e0fb-4d08-98c4-2bcaf123cebb service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:48.328167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:48.328786 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:48.329068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:48.329342 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:48.331202 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:48.346603 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:48.348213 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:48.360877 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1401/2797] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:43:48.757158 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:48.758692 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:48.766976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=100 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:48.864047 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed370c-735e-4f74-b4b2-31f7aaacc05e service cinder] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:48.871275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58361c91-2414-41b0-b983-d8660313534d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:48.872133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58361c91-2414-41b0-b983-d8660313534d service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:48.872706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58361c91-2414-41b0-b983-d8660313534d service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:48.873113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58361c91-2414-41b0-b983-d8660313534d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:48.875817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-58361c91-2414-41b0-b983-d8660313534d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:48.890247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58361c91-2414-41b0-b983-d8660313534d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:48.891848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58361c91-2414-41b0-b983-d8660313534d service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:48.908528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1397/2798] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:43:50.158118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:50.220891 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9249f-8672-4c5b-8545-4ee24d455670 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:50.225385 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:50.226041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:50.226321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:50.226621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] PATH_INFO: `/v3/users/9bb24412ab4a455b859faa8b944d371f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:50.227770 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb24412ab4a455b859faa8b944d371f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:50.229714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:50.277841 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb24412ab4a455b859faa8b944d371f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.289884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb24412ab4a455b859faa8b944d371f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.290568 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb24412ab4a455b859faa8b944d371f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.297665 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb24412ab4a455b859faa8b944d371f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.305250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb24412ab4a455b859faa8b944d371f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.311363 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1402/2799] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:43:50 2022] DELETE /identity/v3/users/9bb24412ab4a455b859faa8b944d371f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:50.315899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86331-034c-4d10-8364-e0a55e147106 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:50.316399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86331-034c-4d10-8364-e0a55e147106 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:50.316526 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86331-034c-4d10-8364-e0a55e147106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:50.340839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-13d86331-034c-4d10-8364-e0a55e147106 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:50.342118 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-13d86331-034c-4d10-8364-e0a55e147106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:50.401786 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1398/2800] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:50.411973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:50.413410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:50.454302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:50.474133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7d727-a93f-472b-9649-1867b46a0302 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:50.478154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b825905-c6af-47ce-b448-8c327a5f1a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:50.478763 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825905-c6af-47ce-b448-8c327a5f1a27 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:50.479047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825905-c6af-47ce-b448-8c327a5f1a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:50.479325 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b825905-c6af-47ce-b448-8c327a5f1a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:50.488707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b825905-c6af-47ce-b448-8c327a5f1a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:50.490228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b825905-c6af-47ce-b448-8c327a5f1a27 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:50.501390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1403/2801] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:43:50.783133 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86331-034c-4d10-8364-e0a55e147106 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:50.784647 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-13d86331-034c-4d10-8364-e0a55e147106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:50.797326 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86331-034c-4d10-8364-e0a55e147106 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:50.802535 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:50.803184 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:50.803459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:50.803744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] PATH_INFO: `/v3/projects/fee4e82230074e3985884e361b46adaf` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:50.806154 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee4e82230074e3985884e361b46adaf)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:50.807772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:50.854565 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee4e82230074e3985884e361b46adaf', 'request_id': 'req-d49b7645-86fd-4651-98b1-a72af97be30b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.860812 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee4e82230074e3985884e361b46adaf', 'request_id': 'req-d49b7645-86fd-4651-98b1-a72af97be30b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.866475 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee4e82230074e3985884e361b46adaf', 'request_id': 'req-d49b7645-86fd-4651-98b1-a72af97be30b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.880235 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] The token cache is being invalidate because project fee4e82230074e3985884e361b46adaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:43:50.880534 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.881181 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:50.882690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1399/2802] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:43:50 2022] DELETE /identity/v3/projects/fee4e82230074e3985884e361b46adaf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:50.886800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b825905-c6af-47ce-b448-8c327a5f1a27 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:50.896530 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b825905-c6af-47ce-b448-8c327a5f1a27 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:50.959330 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:50.959962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:50.960247 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:50.960525 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] PATH_INFO: `/v3/users/96c36d55e61e4a278adef15e355c799c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:50.962064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c36d55e61e4a278adef15e355c799c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:50.963995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:51.011776 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c36d55e61e4a278adef15e355c799c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.024806 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c36d55e61e4a278adef15e355c799c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.025515 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c36d55e61e4a278adef15e355c799c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.032648 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c36d55e61e4a278adef15e355c799c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.040607 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c36d55e61e4a278adef15e355c799c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.046438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1404/2803] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:43:50 2022] DELETE /identity/v3/users/96c36d55e61e4a278adef15e355c799c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:51.322959 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:51.324500 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:51.342885 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b7645-86fd-4651-98b1-a72af97be30b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:51.357530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:51.408175 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:51.408819 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:51.409095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:51.409368 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] PATH_INFO: `/v3/projects/72cc56f86f3e4f2bbb9e71eac3896efb` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:51.410505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=72cc56f86f3e4f2bbb9e71eac3896efb)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:51.412398 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:51.458941 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72cc56f86f3e4f2bbb9e71eac3896efb', 'request_id': 'req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.465592 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72cc56f86f3e4f2bbb9e71eac3896efb', 'request_id': 'req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.471084 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72cc56f86f3e4f2bbb9e71eac3896efb', 'request_id': 'req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.483767 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] The token cache is being invalidate because project 72cc56f86f3e4f2bbb9e71eac3896efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:43:51.484054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.484669 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.486214 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1400/2804] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:43:51 2022] DELETE /identity/v3/projects/72cc56f86f3e4f2bbb9e71eac3896efb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:51.490246 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:51.491650 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:51.505832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42bc1-1ece-4705-9106-48c164b6bb5a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:51.558425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:51.565949 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:51.566672 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:51.567391 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:51.567712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] PATH_INFO: `/v3/users/b1cff275e6154a1c9bf5229ccdd28354` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:51.569267 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1cff275e6154a1c9bf5229ccdd28354)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:51.572287 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:51.619058 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1cff275e6154a1c9bf5229ccdd28354'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.632658 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1cff275e6154a1c9bf5229ccdd28354'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.634534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1cff275e6154a1c9bf5229ccdd28354'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.642027 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1cff275e6154a1c9bf5229ccdd28354'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.650102 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6889896d-c127-44aa-9198-b1a4342d937c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1cff275e6154a1c9bf5229ccdd28354'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:51.656493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1405/2805] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:43:51 2022] DELETE /identity/v3/users/b1cff275e6154a1c9bf5229ccdd28354 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:51.939299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:51.940924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:51.955394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6f41d-ef6c-47d8-8bc1-0fd44a8fa5bb admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.013003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.013588 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.013871 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.014140 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] PATH_INFO: `/v3/projects/854fbd3261074ed786c825fcb8a1a64e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.015248 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] RBAC: Authorizing `identity:delete_project(project_id=854fbd3261074ed786c825fcb8a1a64e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.017177 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.064480 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854fbd3261074ed786c825fcb8a1a64e', 'request_id': 'req-99445de1-81de-4e1d-9728-d679fbf4a97b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.065585 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:52.073268 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854fbd3261074ed786c825fcb8a1a64e', 'request_id': 'req-99445de1-81de-4e1d-9728-d679fbf4a97b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.079390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854fbd3261074ed786c825fcb8a1a64e', 'request_id': 'req-99445de1-81de-4e1d-9728-d679fbf4a97b'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.091017 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] The token cache is being invalidate because project 854fbd3261074ed786c825fcb8a1a64e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:43:52.091307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.091928 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-99445de1-81de-4e1d-9728-d679fbf4a97b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.093390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1401/2806] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:43:51 2022] DELETE /identity/v3/projects/854fbd3261074ed786c825fcb8a1a64e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:52.099427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a952bb-77ac-4ac3-aa82-f884eb494475 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.099721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a952bb-77ac-4ac3-aa82-f884eb494475 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.099999 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a952bb-77ac-4ac3-aa82-f884eb494475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.122072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-06a952bb-77ac-4ac3-aa82-f884eb494475 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:52.123307 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-06a952bb-77ac-4ac3-aa82-f884eb494475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:52.183095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1406/2807] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:52.191580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e785c88-09a6-4cd0-85fa-0b14171ac850 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.191879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e785c88-09a6-4cd0-85fa-0b14171ac850 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.192159 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e785c88-09a6-4cd0-85fa-0b14171ac850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.213332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2e785c88-09a6-4cd0-85fa-0b14171ac850 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:52.214584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2e785c88-09a6-4cd0-85fa-0b14171ac850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:52.222568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1402/2808] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:52.226795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a952bb-77ac-4ac3-aa82-f884eb494475 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:52.236638 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a952bb-77ac-4ac3-aa82-f884eb494475 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.241341 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.241972 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.242286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.242548 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.243201 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:43:52.243506 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.245100 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.257003 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1407/2809] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:43:52.262267 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e785c88-09a6-4cd0-85fa-0b14171ac850 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:52.272004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e785c88-09a6-4cd0-85fa-0b14171ac850 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.278316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58d3cb-f4c9-46d0-ad76-b9eb059dc39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.279043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58d3cb-f4c9-46d0-ad76-b9eb059dc39c admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.279392 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58d3cb-f4c9-46d0-ad76-b9eb059dc39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.279674 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58d3cb-f4c9-46d0-ad76-b9eb059dc39c admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.280332 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58d3cb-f4c9-46d0-ad76-b9eb059dc39c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.282297 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58d3cb-f4c9-46d0-ad76-b9eb059dc39c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.317122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1403/2810] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:43:52 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:43:52.321566 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:52.331600 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deb290-da67-4cbb-b41d-ebd44e4dc85c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.396840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7dbfe-ce42-4eb4-b228-a536eab0d90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.397468 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7dbfe-ce42-4eb4-b228-a536eab0d90b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.397775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7dbfe-ce42-4eb4-b228-a536eab0d90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.398057 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7dbfe-ce42-4eb4-b228-a536eab0d90b admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.398973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7dbfe-ce42-4eb4-b228-a536eab0d90b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.401146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb7dbfe-ce42-4eb4-b228-a536eab0d90b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.411608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:52.428510 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1408/2811] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:43:52 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:43:52.430573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58d3cb-f4c9-46d0-ad76-b9eb059dc39c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:52.441232 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58d3cb-f4c9-46d0-ad76-b9eb059dc39c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.446075 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19768fb-73c9-4b0b-b48c-3722d91dd7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.446689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19768fb-73c9-4b0b-b48c-3722d91dd7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.446974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19768fb-73c9-4b0b-b48c-3722d91dd7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.447253 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19768fb-73c9-4b0b-b48c-3722d91dd7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.447902 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19768fb-73c9-4b0b-b48c-3722d91dd7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.449559 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19768fb-73c9-4b0b-b48c-3722d91dd7ed admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.459874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1404/2812] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:52.464205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7dbfe-ce42-4eb4-b228-a536eab0d90b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:52.474690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7dbfe-ce42-4eb4-b228-a536eab0d90b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.480474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dabd26-7a4e-4ee9-9c37-de904d3575b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.481089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dabd26-7a4e-4ee9-9c37-de904d3575b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.481380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dabd26-7a4e-4ee9-9c37-de904d3575b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.481694 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dabd26-7a4e-4ee9-9c37-de904d3575b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.482352 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dabd26-7a4e-4ee9-9c37-de904d3575b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.483983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dabd26-7a4e-4ee9-9c37-de904d3575b5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.497801 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1409/2813] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:52.502198 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19768fb-73c9-4b0b-b48c-3722d91dd7ed admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:52.513265 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19768fb-73c9-4b0b-b48c-3722d91dd7ed admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.518192 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.518847 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.519125 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.519397 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] PATH_INFO: `/v3/projects/f0bfda72391542aaa06ab975e4d6d088/users/3dedd48a8fac4c40978ef03258e23e88/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.536396 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0bfda72391542aaa06ab975e4d6d088, user_id=3dedd48a8fac4c40978ef03258e23e88, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.539834 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.554131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1405/2814] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:43:52 2022] PUT /identity/v3/projects/f0bfda72391542aaa06ab975e4d6d088/users/3dedd48a8fac4c40978ef03258e23e88/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:43:52.565518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f3824-729f-49b1-a062-4b81566ace01 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.565736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f3824-729f-49b1-a062-4b81566ace01 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.565986 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f3824-729f-49b1-a062-4b81566ace01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.596042 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cd4f3824-729f-49b1-a062-4b81566ace01 None None] MFA Rules not processed for user `3dedd48a8fac4c40978ef03258e23e88`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:52.597390 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cd4f3824-729f-49b1-a062-4b81566ace01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:52.671551 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1410/2815] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:52.684912 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:52.686663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:52.746354 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9076ea7-f87e-442e-b676-dec46fba6d86 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.750325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc13cf-e58f-47b4-9dcd-34fcd8d78b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.750943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc13cf-e58f-47b4-9dcd-34fcd8d78b68 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.751226 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc13cf-e58f-47b4-9dcd-34fcd8d78b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.751503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc13cf-e58f-47b4-9dcd-34fcd8d78b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.760457 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc13cf-e58f-47b4-9dcd-34fcd8d78b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.761931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc13cf-e58f-47b4-9dcd-34fcd8d78b68 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.771103 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1406/2816] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:43:52.792782 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f3824-729f-49b1-a062-4b81566ace01 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:52.856840 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f3824-729f-49b1-a062-4b81566ace01 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:52.861484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:52.862114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.862402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.862684 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] PATH_INFO: `/v3/users/3dedd48a8fac4c40978ef03258e23e88` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:52.863853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dedd48a8fac4c40978ef03258e23e88)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:52.865869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:52.926376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dedd48a8fac4c40978ef03258e23e88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.944394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dedd48a8fac4c40978ef03258e23e88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.945126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dedd48a8fac4c40978ef03258e23e88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.952890 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dedd48a8fac4c40978ef03258e23e88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.955262 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:52.963068 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dedd48a8fac4c40978ef03258e23e88'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:52.972161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1411/2817] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:43:52 2022] DELETE /identity/v3/users/3dedd48a8fac4c40978ef03258e23e88 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:52.977740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005785f-4dff-458d-930e-193494797877 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:52.978034 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005785f-4dff-458d-930e-193494797877 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:52.978675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005785f-4dff-458d-930e-193494797877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.000816 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-1005785f-4dff-458d-930e-193494797877 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:53.002086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1005785f-4dff-458d-930e-193494797877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:53.064727 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1407/2818] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:53.074489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:53.075958 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:53.135513 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0978a1a-32b6-457e-bbe5-61ce9feb9a56 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:53.139251 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c092262-795e-4602-824c-02c2ee68cd40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:53.139863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c092262-795e-4602-824c-02c2ee68cd40 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.140149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c092262-795e-4602-824c-02c2ee68cd40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.140430 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c092262-795e-4602-824c-02c2ee68cd40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.149661 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c092262-795e-4602-824c-02c2ee68cd40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:53.151500 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c092262-795e-4602-824c-02c2ee68cd40 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:53.162320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1412/2819] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:43:53.432973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005785f-4dff-458d-930e-193494797877 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:53.434787 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1005785f-4dff-458d-930e-193494797877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:53.446778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005785f-4dff-458d-930e-193494797877 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:53.451421 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:53.452062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.452345 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.452618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] PATH_INFO: `/v3/projects/f0bfda72391542aaa06ab975e4d6d088` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.453778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0bfda72391542aaa06ab975e4d6d088)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:53.455737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:53.469651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:53.508224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0bfda72391542aaa06ab975e4d6d088', 'request_id': 'req-32168f53-faee-49f0-af43-fd0b2095aed2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:53.514412 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0bfda72391542aaa06ab975e4d6d088', 'request_id': 'req-32168f53-faee-49f0-af43-fd0b2095aed2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:53.520022 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0bfda72391542aaa06ab975e4d6d088', 'request_id': 'req-32168f53-faee-49f0-af43-fd0b2095aed2'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:53.534968 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] The token cache is being invalidate because project f0bfda72391542aaa06ab975e4d6d088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:43:53.535263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:53.535950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-32168f53-faee-49f0-af43-fd0b2095aed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:43:53.537468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1408/2820] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:43:53 2022] DELETE /identity/v3/projects/f0bfda72391542aaa06ab975e4d6d088 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:53.543268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa8370-3b1b-4436-8ec4-233b46c4118d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.543774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa8370-3b1b-4436-8ec4-233b46c4118d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.543874 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa8370-3b1b-4436-8ec4-233b46c4118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.564012 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-39aa8370-3b1b-4436-8ec4-233b46c4118d None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:53.565162 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-39aa8370-3b1b-4436-8ec4-233b46c4118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:53.626312 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1413/2821] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:53.689342 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20a80-557a-4b63-8de0-edb1042c8c16 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.689662 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20a80-557a-4b63-8de0-edb1042c8c16 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.689943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20a80-557a-4b63-8de0-edb1042c8c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.710225 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-55d20a80-557a-4b63-8de0-edb1042c8c16 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:53.712172 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-55d20a80-557a-4b63-8de0-edb1042c8c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:53.723741 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1409/2822] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:53.725081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa8370-3b1b-4436-8ec4-233b46c4118d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:53.735444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa8370-3b1b-4436-8ec4-233b46c4118d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:53.740178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:53.740788 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.741077 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.741358 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.742055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:43:53.742411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:53.744156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:53.756089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1414/2823] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:43:53.760976 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d20a80-557a-4b63-8de0-edb1042c8c16 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:53.770284 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d20a80-557a-4b63-8de0-edb1042c8c16 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:53.775720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eecf4-1291-4a67-a229-01b38a451ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:53.776319 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eecf4-1291-4a67-a229-01b38a451ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.776604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eecf4-1291-4a67-a229-01b38a451ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.776878 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4eecf4-1291-4a67-a229-01b38a451ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.777580 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eecf4-1291-4a67-a229-01b38a451ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:53.779468 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4eecf4-1291-4a67-a229-01b38a451ce9 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:53.807055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1410/2824] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:43:53 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:43:53.811215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:53.822666 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b37145-4aeb-4abf-a6df-188805bac506 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:53.882349 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d392532-e16d-4799-be8c-adfa2fbaffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:53.883460 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d392532-e16d-4799-be8c-adfa2fbaffa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.883742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d392532-e16d-4799-be8c-adfa2fbaffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.884141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d392532-e16d-4799-be8c-adfa2fbaffa7 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.884795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d392532-e16d-4799-be8c-adfa2fbaffa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:53.886748 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d392532-e16d-4799-be8c-adfa2fbaffa7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:53.908677 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1415/2825] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:43:53 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:43:53.912887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eecf4-1291-4a67-a229-01b38a451ce9 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:53.923444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4eecf4-1291-4a67-a229-01b38a451ce9 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:53.928072 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246958c-856c-4563-9dcc-81245e8bc7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:53.928668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246958c-856c-4563-9dcc-81245e8bc7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.928950 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246958c-856c-4563-9dcc-81245e8bc7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.929221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246958c-856c-4563-9dcc-81245e8bc7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.929958 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246958c-856c-4563-9dcc-81245e8bc7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:53.931547 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246958c-856c-4563-9dcc-81245e8bc7e5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:53.941316 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1411/2826] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:53.946055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d392532-e16d-4799-be8c-adfa2fbaffa7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:53.956367 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d392532-e16d-4799-be8c-adfa2fbaffa7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:53.961969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e00624-8a29-432c-a404-159df44c90a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:53.962567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e00624-8a29-432c-a404-159df44c90a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.962861 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e00624-8a29-432c-a404-159df44c90a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.963131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e00624-8a29-432c-a404-159df44c90a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:53.963817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e00624-8a29-432c-a404-159df44c90a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:53.965505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e00624-8a29-432c-a404-159df44c90a7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:53.970928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:43:53.979286 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1416/2827] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:43:53.980467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246958c-856c-4563-9dcc-81245e8bc7e5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:53.991309 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246958c-856c-4563-9dcc-81245e8bc7e5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:53.996045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:53.996699 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:53.997062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:53.997250 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] PATH_INFO: `/v3/projects/5009325c0ea04fecb97f767bc1418474/users/5177dca8e168401a82cd75b788bf7e8d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:54.012804 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5009325c0ea04fecb97f767bc1418474, user_id=5177dca8e168401a82cd75b788bf7e8d, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:54.015499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:54.030650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1412/2828] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:43:53 2022] PUT /identity/v3/projects/5009325c0ea04fecb97f767bc1418474/users/5177dca8e168401a82cd75b788bf7e8d/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:43:54.040749 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194203f7-2ccb-4669-8888-ffe05bffd839 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:54.041033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194203f7-2ccb-4669-8888-ffe05bffd839 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:54.041306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194203f7-2ccb-4669-8888-ffe05bffd839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:54.072053 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-194203f7-2ccb-4669-8888-ffe05bffd839 None None] MFA Rules not processed for user `5177dca8e168401a82cd75b788bf7e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:43:54.073716 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-194203f7-2ccb-4669-8888-ffe05bffd839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:54.141368 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1417/2829] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:43:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:43:54.153921 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:43:54.155611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:43:54.213897 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7dbaf-694f-40e2-a6c7-348d35cd90b8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:43:54.217595 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684cee4-ef52-4082-9d12-c363f8b39bfe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '464190e0ccaf4f82942f518df1bf6c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:43:54.218209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684cee4-ef52-4082-9d12-c363f8b39bfe service glance] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:43:54.218495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684cee4-ef52-4082-9d12-c363f8b39bfe service glance] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:43:54.218772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684cee4-ef52-4082-9d12-c363f8b39bfe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:43:54.227359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684cee4-ef52-4082-9d12-c363f8b39bfe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:43:54.228798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684cee4-ef52-4082-9d12-c363f8b39bfe service glance] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:43:54.237578 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1413/2830] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:43:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:44:04.138323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=32 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:44:04.233613 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:44:28.818006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194203f7-2ccb-4669-8888-ffe05bffd839 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:28.881230 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194203f7-2ccb-4669-8888-ffe05bffd839 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:28.885859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:28.886493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:28.886777 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:28.887055 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] PATH_INFO: `/v3/users/5177dca8e168401a82cd75b788bf7e8d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:28.888188 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5177dca8e168401a82cd75b788bf7e8d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:28.890209 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:28.944250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5177dca8e168401a82cd75b788bf7e8d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:28.957361 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5177dca8e168401a82cd75b788bf7e8d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:28.958128 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5177dca8e168401a82cd75b788bf7e8d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:28.966631 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5177dca8e168401a82cd75b788bf7e8d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:28.974435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5177dca8e168401a82cd75b788bf7e8d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:28.980797 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1418/2831] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:44:28 2022] DELETE /identity/v3/users/5177dca8e168401a82cd75b788bf7e8d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:44:28.986121 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d372f-7d5c-4f57-8eaa-1d2b02b8d4a3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:28.986655 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d372f-7d5c-4f57-8eaa-1d2b02b8d4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:28.986655 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d372f-7d5c-4f57-8eaa-1d2b02b8d4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.006260 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ca7d372f-7d5c-4f57-8eaa-1d2b02b8d4a3 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:44:29.007557 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ca7d372f-7d5c-4f57-8eaa-1d2b02b8d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:44:29.068131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1414/2832] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:44:29.079163 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:29.080832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:44:29.141895 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e27f4-d299-419b-9944-6e867758d0b2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:29.145759 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47736a7a-b3d7-4007-a1e2-efa54ae2f08d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:29.146616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736a7a-b3d7-4007-a1e2-efa54ae2f08d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.146774 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736a7a-b3d7-4007-a1e2-efa54ae2f08d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.146929 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736a7a-b3d7-4007-a1e2-efa54ae2f08d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.157191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47736a7a-b3d7-4007-a1e2-efa54ae2f08d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:29.158693 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47736a7a-b3d7-4007-a1e2-efa54ae2f08d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:29.168925 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1419/2833] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:44:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:44:29.464484 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d372f-7d5c-4f57-8eaa-1d2b02b8d4a3 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:29.466043 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ca7d372f-7d5c-4f57-8eaa-1d2b02b8d4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:44:29.479627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d372f-7d5c-4f57-8eaa-1d2b02b8d4a3 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:29.484966 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:29.485704 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.485997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.486291 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] PATH_INFO: `/v3/projects/5009325c0ea04fecb97f767bc1418474` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.487479 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5009325c0ea04fecb97f767bc1418474)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:29.489434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:29.543070 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5009325c0ea04fecb97f767bc1418474', 'request_id': 'req-4e0ce7a5-57e6-4036-82af-c7624ffd6770'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:29.549846 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5009325c0ea04fecb97f767bc1418474', 'request_id': 'req-4e0ce7a5-57e6-4036-82af-c7624ffd6770'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:29.555492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5009325c0ea04fecb97f767bc1418474', 'request_id': 'req-4e0ce7a5-57e6-4036-82af-c7624ffd6770'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:29.572525 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] The token cache is being invalidate because project 5009325c0ea04fecb97f767bc1418474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:44:29.572982 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:29.574063 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-4e0ce7a5-57e6-4036-82af-c7624ffd6770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:44:29.576251 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1415/2834] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:44:29 2022] DELETE /identity/v3/projects/5009325c0ea04fecb97f767bc1418474 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:44:29.582015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c8881-c26d-4dd5-8128-d72f8622136c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.582381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c8881-c26d-4dd5-8128-d72f8622136c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.582727 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c8881-c26d-4dd5-8128-d72f8622136c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.603973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0b2c8881-c26d-4dd5-8128-d72f8622136c None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:44:29.605203 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0b2c8881-c26d-4dd5-8128-d72f8622136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:44:29.625117 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:44:29.670272 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1420/2835] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:44:29.679459 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2b0de-3939-4568-a100-7136cfcb2c78 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.679750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2b0de-3939-4568-a100-7136cfcb2c78 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.680031 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2b0de-3939-4568-a100-7136cfcb2c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.700879 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-ede2b0de-3939-4568-a100-7136cfcb2c78 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:44:29.702924 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-ede2b0de-3939-4568-a100-7136cfcb2c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:44:29.711568 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1416/2836] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:44:29.716237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c8881-c26d-4dd5-8128-d72f8622136c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:29.726601 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c8881-c26d-4dd5-8128-d72f8622136c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:29.731404 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:29.732013 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.732302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.732608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.733332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:44:29.733721 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:29.735439 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:29.748295 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1421/2837] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:44:29.754199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2b0de-3939-4568-a100-7136cfcb2c78 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:29.763701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2b0de-3939-4568-a100-7136cfcb2c78 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:29.769644 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87247095-0bbc-466d-936c-b0f07d6df7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:29.770274 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87247095-0bbc-466d-936c-b0f07d6df7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.770567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87247095-0bbc-466d-936c-b0f07d6df7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.770835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87247095-0bbc-466d-936c-b0f07d6df7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.771485 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87247095-0bbc-466d-936c-b0f07d6df7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:29.773381 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87247095-0bbc-466d-936c-b0f07d6df7e3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:29.802441 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1417/2838] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:44:29 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:44:29.806714 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:29.817820 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bba60-e061-44b4-9b7c-9c28ca741cca admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:29.880467 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2eb53-3a7d-4575-881e-61a1bb0ca55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:29.881093 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2eb53-3a7d-4575-881e-61a1bb0ca55a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.881382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2eb53-3a7d-4575-881e-61a1bb0ca55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.881697 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2eb53-3a7d-4575-881e-61a1bb0ca55a admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.882347 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2eb53-3a7d-4575-881e-61a1bb0ca55a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:29.885486 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2eb53-3a7d-4575-881e-61a1bb0ca55a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:29.908660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1422/2839] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:44:29 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:44:29.913993 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87247095-0bbc-466d-936c-b0f07d6df7e3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:29.925093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87247095-0bbc-466d-936c-b0f07d6df7e3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:29.930365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ace66e-366a-46c0-bbe7-adb9e5bd0b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:29.931111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace66e-366a-46c0-bbe7-adb9e5bd0b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.931491 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace66e-366a-46c0-bbe7-adb9e5bd0b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.931872 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ace66e-366a-46c0-bbe7-adb9e5bd0b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.932632 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ace66e-366a-46c0-bbe7-adb9e5bd0b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:29.934460 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ace66e-366a-46c0-bbe7-adb9e5bd0b0f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:29.945914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1418/2840] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:44:29.950249 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2eb53-3a7d-4575-881e-61a1bb0ca55a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:29.960579 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2eb53-3a7d-4575-881e-61a1bb0ca55a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:29.965237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8e704-e292-4a2a-8222-a0996ec781ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:29.965884 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8e704-e292-4a2a-8222-a0996ec781ce admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:29.966174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8e704-e292-4a2a-8222-a0996ec781ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:29.966459 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8e704-e292-4a2a-8222-a0996ec781ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:29.967113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8e704-e292-4a2a-8222-a0996ec781ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:29.968795 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8e704-e292-4a2a-8222-a0996ec781ce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:29.979462 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1423/2841] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:44:29.984135 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ace66e-366a-46c0-bbe7-adb9e5bd0b0f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:29.994254 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ace66e-366a-46c0-bbe7-adb9e5bd0b0f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:29.999990 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:30.000672 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:30.001003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:30.001276 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] PATH_INFO: `/v3/projects/bec83283f3554b5b886349efecc6986e/users/77bcb54aeb9f411b9a223e4728427dcd/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:30.016041 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec83283f3554b5b886349efecc6986e, user_id=77bcb54aeb9f411b9a223e4728427dcd, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:30.018718 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:30.031567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1419/2842] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:44:29 2022] PUT /identity/v3/projects/bec83283f3554b5b886349efecc6986e/users/77bcb54aeb9f411b9a223e4728427dcd/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:44:30.041899 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493eb1e-0624-458d-8b1b-b97794fab49b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:30.042192 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493eb1e-0624-458d-8b1b-b97794fab49b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:30.042487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493eb1e-0624-458d-8b1b-b97794fab49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:30.071228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d493eb1e-0624-458d-8b1b-b97794fab49b None None] MFA Rules not processed for user `77bcb54aeb9f411b9a223e4728427dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:44:30.072452 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d493eb1e-0624-458d-8b1b-b97794fab49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:44:30.136285 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1424/2843] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:44:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:44:30.148053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:44:30.149770 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:44:30.172822 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:44:30.211390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7844ecb-dd55-4020-84af-2707c1e3fd06 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:44:30.215087 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a719ce-2ea7-428f-a386-c0d92c32bade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:44:30.215689 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a719ce-2ea7-428f-a386-c0d92c32bade service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:44:30.215979 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a719ce-2ea7-428f-a386-c0d92c32bade service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:44:30.216256 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a719ce-2ea7-428f-a386-c0d92c32bade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:44:30.230095 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a719ce-2ea7-428f-a386-c0d92c32bade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:44:30.231492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a719ce-2ea7-428f-a386-c0d92c32bade service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:44:30.240678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1420/2844] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:44:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:44:40.133376 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:44:40.236630 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=17 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:07.847587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493eb1e-0624-458d-8b1b-b97794fab49b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:07.917083 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493eb1e-0624-458d-8b1b-b97794fab49b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:07.921671 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:07.922319 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:07.922608 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:07.922890 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] PATH_INFO: `/v3/users/77bcb54aeb9f411b9a223e4728427dcd` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:07.924081 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77bcb54aeb9f411b9a223e4728427dcd)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:07.926110 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:07.977474 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77bcb54aeb9f411b9a223e4728427dcd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:07.989751 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77bcb54aeb9f411b9a223e4728427dcd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:07.990447 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77bcb54aeb9f411b9a223e4728427dcd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:07.998373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77bcb54aeb9f411b9a223e4728427dcd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:08.005787 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77bcb54aeb9f411b9a223e4728427dcd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:08.011893 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1425/2845] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:45:07 2022] DELETE /identity/v3/users/77bcb54aeb9f411b9a223e4728427dcd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:45:08.016617 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23526-3e1e-4fc6-b6c2-775f60a1a5e6 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.017112 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23526-3e1e-4fc6-b6c2-775f60a1a5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.017212 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb23526-3e1e-4fc6-b6c2-775f60a1a5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.036176 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5eb23526-3e1e-4fc6-b6c2-775f60a1a5e6 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:08.037397 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5eb23526-3e1e-4fc6-b6c2-775f60a1a5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:08.097513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1421/2846] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:08.107922 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:08.109380 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:08.167719 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a382978-325e-4968-b655-6d6932fab871 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:08.171306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602406-8a95-4b5d-8418-8bb8415a4961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:08.171911 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602406-8a95-4b5d-8418-8bb8415a4961 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.172191 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602406-8a95-4b5d-8418-8bb8415a4961 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.172464 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602406-8a95-4b5d-8418-8bb8415a4961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.182154 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1602406-8a95-4b5d-8418-8bb8415a4961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:08.183582 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1602406-8a95-4b5d-8418-8bb8415a4961 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:08.194854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1426/2847] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:08.469320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb23526-3e1e-4fc6-b6c2-775f60a1a5e6 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:08.470815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5eb23526-3e1e-4fc6-b6c2-775f60a1a5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:08.485518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb23526-3e1e-4fc6-b6c2-775f60a1a5e6 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:08.490424 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:08.491157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.491440 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.491713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] PATH_INFO: `/v3/projects/bec83283f3554b5b886349efecc6986e` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.493444 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec83283f3554b5b886349efecc6986e)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:08.495399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:08.545029 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec83283f3554b5b886349efecc6986e', 'request_id': 'req-a856e7db-f6ea-4761-91cc-ad70ef24ea09'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:08.551862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec83283f3554b5b886349efecc6986e', 'request_id': 'req-a856e7db-f6ea-4761-91cc-ad70ef24ea09'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:08.558178 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec83283f3554b5b886349efecc6986e', 'request_id': 'req-a856e7db-f6ea-4761-91cc-ad70ef24ea09'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:08.571320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] The token cache is being invalidate because project bec83283f3554b5b886349efecc6986e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:45:08.571611 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:08.572237 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-a856e7db-f6ea-4761-91cc-ad70ef24ea09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:08.573745 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1422/2848] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:45:08 2022] DELETE /identity/v3/projects/bec83283f3554b5b886349efecc6986e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:45:08.578692 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0f9e-6b1b-4b01-9d77-1ff5db62f6c0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.578983 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0f9e-6b1b-4b01-9d77-1ff5db62f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.579258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c0f9e-6b1b-4b01-9d77-1ff5db62f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.600112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-377c0f9e-6b1b-4b01-9d77-1ff5db62f6c0 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:08.601859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-377c0f9e-6b1b-4b01-9d77-1ff5db62f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:08.619863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:08.665475 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1427/2849] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:08.673016 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8529f-e93d-4499-8598-2b18bc878078 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.673307 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8529f-e93d-4499-8598-2b18bc878078 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.673615 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8529f-e93d-4499-8598-2b18bc878078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.693987 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a5b8529f-e93d-4499-8598-2b18bc878078 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:08.695340 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a5b8529f-e93d-4499-8598-2b18bc878078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:08.703275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1423/2850] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:08.707493 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c0f9e-6b1b-4b01-9d77-1ff5db62f6c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:08.718329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c0f9e-6b1b-4b01-9d77-1ff5db62f6c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:08.723001 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:08.723620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.723904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.724181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.724844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:45:08.725155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:08.727394 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:08.739839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1428/2851] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:45:08.745213 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8529f-e93d-4499-8598-2b18bc878078 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:08.754931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8529f-e93d-4499-8598-2b18bc878078 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:08.762281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b637a2-b568-42bc-93fa-f10927a2c751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:08.762985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b637a2-b568-42bc-93fa-f10927a2c751 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.763366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b637a2-b568-42bc-93fa-f10927a2c751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.763713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b637a2-b568-42bc-93fa-f10927a2c751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.764369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b637a2-b568-42bc-93fa-f10927a2c751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:08.766364 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b637a2-b568-42bc-93fa-f10927a2c751 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:08.794552 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1424/2852] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:45:08 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:45:08.798715 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:08.809894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc48ef6-20b1-4967-b8db-32889a421d17 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:08.871379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a2f27-3626-48c7-a48e-38b9315204f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:08.872080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a2f27-3626-48c7-a48e-38b9315204f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.872269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a2f27-3626-48c7-a48e-38b9315204f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.872549 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a2f27-3626-48c7-a48e-38b9315204f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.873302 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a2f27-3626-48c7-a48e-38b9315204f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:08.877332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a2f27-3626-48c7-a48e-38b9315204f8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:08.903410 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1429/2853] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:45:08 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:45:08.904857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b637a2-b568-42bc-93fa-f10927a2c751 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:08.915874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b637a2-b568-42bc-93fa-f10927a2c751 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:08.920553 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1df0a-5df8-4fc5-b6c1-0a97f72baac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:08.921166 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1df0a-5df8-4fc5-b6c1-0a97f72baac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.921478 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1df0a-5df8-4fc5-b6c1-0a97f72baac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.921753 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1df0a-5df8-4fc5-b6c1-0a97f72baac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.922397 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1df0a-5df8-4fc5-b6c1-0a97f72baac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:08.924002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1df0a-5df8-4fc5-b6c1-0a97f72baac7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:08.935130 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1425/2854] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:45:08.939237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a2f27-3626-48c7-a48e-38b9315204f8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:08.949422 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a2f27-3626-48c7-a48e-38b9315204f8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:08.954144 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fbfd9-dab6-4aa1-ac06-eaf965318222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:08.954741 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fbfd9-dab6-4aa1-ac06-eaf965318222 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.955033 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fbfd9-dab6-4aa1-ac06-eaf965318222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.955318 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fbfd9-dab6-4aa1-ac06-eaf965318222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:08.955966 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9fbfd9-dab6-4aa1-ac06-eaf965318222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:08.957619 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9fbfd9-dab6-4aa1-ac06-eaf965318222 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:08.968438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1430/2855] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:45:08.972597 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1df0a-5df8-4fc5-b6c1-0a97f72baac7 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:08.983160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1df0a-5df8-4fc5-b6c1-0a97f72baac7 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:08.988288 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:08.989160 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:08.989442 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:08.989749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] PATH_INFO: `/v3/projects/7bd22634160a4ad38a7e57b11bd0c2db/users/a8c5959bcd8a4909b94e7253676a5eac/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:09.005379 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd22634160a4ad38a7e57b11bd0c2db, user_id=a8c5959bcd8a4909b94e7253676a5eac, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:09.008165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:09.022773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1426/2856] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:45:08 2022] PUT /identity/v3/projects/7bd22634160a4ad38a7e57b11bd0c2db/users/a8c5959bcd8a4909b94e7253676a5eac/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:45:09.033181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a0fb0-0971-4698-bfdd-6182bdee875b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:09.033498 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a0fb0-0971-4698-bfdd-6182bdee875b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:09.033781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a0fb0-0971-4698-bfdd-6182bdee875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:09.061707 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1c1a0fb0-0971-4698-bfdd-6182bdee875b None None] MFA Rules not processed for user `a8c5959bcd8a4909b94e7253676a5eac`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:09.062915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1c1a0fb0-0971-4698-bfdd-6182bdee875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:09.127409 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1431/2857] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:09.139321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:09.141008 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:09.166082 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:09.204306 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e06b8-28bb-4cb8-b5ba-0e00478cc4e8 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:09.207917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fb471-7ba1-44d6-ab5f-6c4931cf2d36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:09.208532 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fb471-7ba1-44d6-ab5f-6c4931cf2d36 service cinder] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:09.208815 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fb471-7ba1-44d6-ab5f-6c4931cf2d36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:09.209092 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fb471-7ba1-44d6-ab5f-6c4931cf2d36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:09.217962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fb471-7ba1-44d6-ab5f-6c4931cf2d36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:09.219453 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fb471-7ba1-44d6-ab5f-6c4931cf2d36 service cinder] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:09.228957 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1427/2858] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:10.610546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef70fa5-46bb-4d08-b079-1837868e7ea3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:10.611072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef70fa5-46bb-4d08-b079-1837868e7ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:10.611072 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef70fa5-46bb-4d08-b079-1837868e7ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:10.630802 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bef70fa5-46bb-4d08-b079-1837868e7ea3 None None] MFA Rules not processed for user `a8c5959bcd8a4909b94e7253676a5eac`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:10.632029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bef70fa5-46bb-4d08-b079-1837868e7ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:10.640340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1432/2859] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:19.224209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=17 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:20.625988 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:30.021887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9200b9-3c45-49d3-ba33-a9765f37e0f3 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:30.022434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9200b9-3c45-49d3-ba33-a9765f37e0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:30.022434 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9200b9-3c45-49d3-ba33-a9765f37e0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:30.041044 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-0e9200b9-3c45-49d3-ba33-a9765f37e0f3 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:30.042271 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0e9200b9-3c45-49d3-ba33-a9765f37e0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:30.098025 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1428/2860] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:30.147880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef70fa5-46bb-4d08-b079-1837868e7ea3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:30.149340 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bef70fa5-46bb-4d08-b079-1837868e7ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:30.225049 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef70fa5-46bb-4d08-b079-1837868e7ea3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:30.230141 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e5eaa-a50c-4cd2-9f35-8a53cb002971 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:30.230758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5eaa-a50c-4cd2-9f35-8a53cb002971 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:30.231047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5eaa-a50c-4cd2-9f35-8a53cb002971 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:30.231331 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e5eaa-a50c-4cd2-9f35-8a53cb002971 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:30.240109 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e5eaa-a50c-4cd2-9f35-8a53cb002971 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:30.241737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e5eaa-a50c-4cd2-9f35-8a53cb002971 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:30.250758 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1433/2861] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:45:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:45:30.772091 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3529993-eef0-4eaf-a0e2-5b4c8751e2fb None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:30.772383 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3529993-eef0-4eaf-a0e2-5b4c8751e2fb None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:30.772749 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3529993-eef0-4eaf-a0e2-5b4c8751e2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:30.792539 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-a3529993-eef0-4eaf-a0e2-5b4c8751e2fb None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:30.793887 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-a3529993-eef0-4eaf-a0e2-5b4c8751e2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:30.801033 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1429/2862] 10.222.0.58 () {58 vars in 967 bytes} [Mon Oct 3 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:30.813971 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e5eaa-a50c-4cd2-9f35-8a53cb002971 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:30.824618 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e5eaa-a50c-4cd2-9f35-8a53cb002971 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:30.829778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ed8f5-8734-43cf-9285-f43d4b691a25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:30.830379 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ed8f5-8734-43cf-9285-f43d4b691a25 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:30.830662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ed8f5-8734-43cf-9285-f43d4b691a25 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:30.830937 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ed8f5-8734-43cf-9285-f43d4b691a25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:30.838962 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ed8f5-8734-43cf-9285-f43d4b691a25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:30.840528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ed8f5-8734-43cf-9285-f43d4b691a25 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:30.849211 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1434/2863] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:45:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:45:31.079299 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9c944-63bd-424a-b857-476fcb8f2e37 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:31.079829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9c944-63bd-424a-b857-476fcb8f2e37 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:31.079953 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9c944-63bd-424a-b857-476fcb8f2e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:31.099074 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-77f9c944-63bd-424a-b857-476fcb8f2e37 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:31.100269 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-77f9c944-63bd-424a-b857-476fcb8f2e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:31.108182 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1430/2864] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:45:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:31.120659 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ed8f5-8734-43cf-9285-f43d4b691a25 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:31.132150 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ed8f5-8734-43cf-9285-f43d4b691a25 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:31.137480 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa827a9e-9554-4eb3-a8a0-89c3de401251 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:31.138095 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa827a9e-9554-4eb3-a8a0-89c3de401251 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:31.138532 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa827a9e-9554-4eb3-a8a0-89c3de401251 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:31.138834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa827a9e-9554-4eb3-a8a0-89c3de401251 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:31.147290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa827a9e-9554-4eb3-a8a0-89c3de401251 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:31.148875 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa827a9e-9554-4eb3-a8a0-89c3de401251 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:31.157093 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1435/2865] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:45:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:45:31.486257 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f384a57-0df5-4265-b0db-9b47891b1296 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:31.486544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f384a57-0df5-4265-b0db-9b47891b1296 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:31.486824 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f384a57-0df5-4265-b0db-9b47891b1296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:31.507501 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-8f384a57-0df5-4265-b0db-9b47891b1296 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:31.508707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8f384a57-0df5-4265-b0db-9b47891b1296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:31.516542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1431/2866] 10.222.0.58 () {58 vars in 967 bytes} [Mon Oct 3 13:45:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:31.529016 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa827a9e-9554-4eb3-a8a0-89c3de401251 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:31.540069 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa827a9e-9554-4eb3-a8a0-89c3de401251 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:31.545727 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bf61b-9c4c-4db9-8e72-44f747be7c42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:31.546338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bf61b-9c4c-4db9-8e72-44f747be7c42 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:31.546627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bf61b-9c4c-4db9-8e72-44f747be7c42 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:31.546909 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bf61b-9c4c-4db9-8e72-44f747be7c42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:31.555218 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bf61b-9c4c-4db9-8e72-44f747be7c42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:31.556807 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bf61b-9c4c-4db9-8e72-44f747be7c42 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:31.565610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1436/2867] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:45:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:45:33.897740 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f384a57-0df5-4265-b0db-9b47891b1296 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:33.960734 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f384a57-0df5-4265-b0db-9b47891b1296 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:33.965165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:33.965825 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:33.966104 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:33.966415 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] PATH_INFO: `/v3/users/a8c5959bcd8a4909b94e7253676a5eac` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:33.967715 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c5959bcd8a4909b94e7253676a5eac)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:33.969649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.016509 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c5959bcd8a4909b94e7253676a5eac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.017210 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c5959bcd8a4909b94e7253676a5eac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.017842 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c5959bcd8a4909b94e7253676a5eac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.024721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c5959bcd8a4909b94e7253676a5eac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.036978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c5959bcd8a4909b94e7253676a5eac'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.051917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1432/2868] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:45:33 2022] DELETE /identity/v3/users/a8c5959bcd8a4909b94e7253676a5eac => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:45:34.056718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c0ec-b0e8-4aeb-a602-288d0414b938 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.057388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c0ec-b0e8-4aeb-a602-288d0414b938 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.057388 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c0ec-b0e8-4aeb-a602-288d0414b938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.077323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9745c0ec-b0e8-4aeb-a602-288d0414b938 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:34.078588 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9745c0ec-b0e8-4aeb-a602-288d0414b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:34.146366 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1437/2869] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:34.157139 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:34.158676 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:34.165376 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:34.222616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d47860-9810-41a4-b7c9-bc9b84091dc5 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:34.226418 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10319336-c589-46e2-a5ba-764e966eee54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:34.227066 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10319336-c589-46e2-a5ba-764e966eee54 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.227352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10319336-c589-46e2-a5ba-764e966eee54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.227684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10319336-c589-46e2-a5ba-764e966eee54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.237944 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10319336-c589-46e2-a5ba-764e966eee54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:34.239419 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10319336-c589-46e2-a5ba-764e966eee54 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.250622 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1433/2870] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:34.432152 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745c0ec-b0e8-4aeb-a602-288d0414b938 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:34.433695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9745c0ec-b0e8-4aeb-a602-288d0414b938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:34.449155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745c0ec-b0e8-4aeb-a602-288d0414b938 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:34.453948 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:34.454596 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.454880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.455156 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] PATH_INFO: `/v3/projects/7bd22634160a4ad38a7e57b11bd0c2db` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.456300 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd22634160a4ad38a7e57b11bd0c2db)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:34.458408 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.508756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:34.512281 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd22634160a4ad38a7e57b11bd0c2db', 'request_id': 'req-e1250d8e-c829-4d0c-a758-fd576c6c663f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.518894 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd22634160a4ad38a7e57b11bd0c2db', 'request_id': 'req-e1250d8e-c829-4d0c-a758-fd576c6c663f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.527320 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd22634160a4ad38a7e57b11bd0c2db', 'request_id': 'req-e1250d8e-c829-4d0c-a758-fd576c6c663f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.540746 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] The token cache is being invalidate because project 7bd22634160a4ad38a7e57b11bd0c2db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:45:34.541047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.541739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1250d8e-c829-4d0c-a758-fd576c6c663f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:45:34.543239 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1438/2871] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:45:34 2022] DELETE /identity/v3/projects/7bd22634160a4ad38a7e57b11bd0c2db => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:45:34.549099 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7651e-b0ca-4353-b9c5-ee79b0be9663 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.549371 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7651e-b0ca-4353-b9c5-ee79b0be9663 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.549668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7651e-b0ca-4353-b9c5-ee79b0be9663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.569132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-12c7651e-b0ca-4353-b9c5-ee79b0be9663 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:34.570370 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-12c7651e-b0ca-4353-b9c5-ee79b0be9663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:34.632899 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1434/2872] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:34.641029 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca17d4-a867-467a-8f0e-98a496b18396 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.641409 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca17d4-a867-467a-8f0e-98a496b18396 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.641728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca17d4-a867-467a-8f0e-98a496b18396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.662070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f6ca17d4-a867-467a-8f0e-98a496b18396 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:34.663290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f6ca17d4-a867-467a-8f0e-98a496b18396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:34.672006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1439/2873] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:34.676463 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7651e-b0ca-4353-b9c5-ee79b0be9663 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:34.686655 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7651e-b0ca-4353-b9c5-ee79b0be9663 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:34.691451 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:34.692050 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.692330 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.692599 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.693239 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81423) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:45:34.693575 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:34.695143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.706694 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1435/2874] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 03 13:45:34.710994 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca17d4-a867-467a-8f0e-98a496b18396 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:34.721112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca17d4-a867-467a-8f0e-98a496b18396 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:34.725915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82658ad5-e4ed-4a50-bb05-c43ed3111218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:34.726518 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82658ad5-e4ed-4a50-bb05-c43ed3111218 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.726800 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82658ad5-e4ed-4a50-bb05-c43ed3111218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.727075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82658ad5-e4ed-4a50-bb05-c43ed3111218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.727710 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82658ad5-e4ed-4a50-bb05-c43ed3111218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:34.729655 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82658ad5-e4ed-4a50-bb05-c43ed3111218 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.757585 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1440/2875] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:45:34 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:45:34.762014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:34.772576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb9f49-4146-4e25-9875-b6bce2ba6a72 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:34.829086 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb462c-756e-467a-8a56-f10db47b6292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:34.829732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb462c-756e-467a-8a56-f10db47b6292 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.830014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb462c-756e-467a-8a56-f10db47b6292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.830429 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb462c-756e-467a-8a56-f10db47b6292 admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.831059 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb462c-756e-467a-8a56-f10db47b6292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:34.832970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb462c-756e-467a-8a56-f10db47b6292 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.855515 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1436/2876] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:45:34 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:45:34.860728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82658ad5-e4ed-4a50-bb05-c43ed3111218 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:34.871107 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82658ad5-e4ed-4a50-bb05-c43ed3111218 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:34.876824 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bebb15-63d8-4c92-8da8-f67feddbca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:34.877426 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bebb15-63d8-4c92-8da8-f67feddbca32 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.877742 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bebb15-63d8-4c92-8da8-f67feddbca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.878023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bebb15-63d8-4c92-8da8-f67feddbca32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.878679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bebb15-63d8-4c92-8da8-f67feddbca32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:34.880273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bebb15-63d8-4c92-8da8-f67feddbca32 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.892664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1441/2877] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:45:34.896904 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb462c-756e-467a-8a56-f10db47b6292 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:34.907649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb462c-756e-467a-8a56-f10db47b6292 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:34.912706 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74773f-7c05-4948-aa74-76861f0ad8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:34.913321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74773f-7c05-4948-aa74-76861f0ad8ea admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.913651 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74773f-7c05-4948-aa74-76861f0ad8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.913931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74773f-7c05-4948-aa74-76861f0ad8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.914582 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74773f-7c05-4948-aa74-76861f0ad8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:34.916280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74773f-7c05-4948-aa74-76861f0ad8ea admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.927961 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1437/2878] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 03 13:45:34.932323 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bebb15-63d8-4c92-8da8-f67feddbca32 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:34.942864 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bebb15-63d8-4c92-8da8-f67feddbca32 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:34.947602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:34.948256 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.948543 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.948825 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] PATH_INFO: `/v3/projects/e8b666774ba8449f90cf329b90725fe4/users/24f16ea7c1d740ac8de5a4a19b165350/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:34.963587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8b666774ba8449f90cf329b90725fe4, user_id=24f16ea7c1d740ac8de5a4a19b165350, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:34.966332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:34.981598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1442/2879] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:45:34 2022] PUT /identity/v3/projects/e8b666774ba8449f90cf329b90725fe4/users/24f16ea7c1d740ac8de5a4a19b165350/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 03 13:45:34.987664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b368a-7624-4dea-93d7-f18f1822e193 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:34.987952 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b368a-7624-4dea-93d7-f18f1822e193 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:34.988221 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b368a-7624-4dea-93d7-f18f1822e193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:35.007683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-547b368a-7624-4dea-93d7-f18f1822e193 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:35.008886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-547b368a-7624-4dea-93d7-f18f1822e193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:35.017061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:35.073551 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1438/2880] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:35.083612 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:35.085484 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:35.143400 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5db8d3-8ad4-45d8-946a-32cf927f6ea9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:35.147314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0026df-4888-4fc9-a633-66a6a6410a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:35.147918 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0026df-4888-4fc9-a633-66a6a6410a14 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:35.148205 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0026df-4888-4fc9-a633-66a6a6410a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:35.148487 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0026df-4888-4fc9-a633-66a6a6410a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:35.158121 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0026df-4888-4fc9-a633-66a6a6410a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:35.159562 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0026df-4888-4fc9-a633-66a6a6410a14 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:35.169471 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1443/2881] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:39.349132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95d45-4984-487a-bef5-15670fdbc83a None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:39.349720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95d45-4984-487a-bef5-15670fdbc83a None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:39.349720 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e95d45-4984-487a-bef5-15670fdbc83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:39.384972 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-d1e95d45-4984-487a-bef5-15670fdbc83a None None] MFA Rules not processed for user `24f16ea7c1d740ac8de5a4a19b165350`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:39.386320 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d1e95d45-4984-487a-bef5-15670fdbc83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:39.458554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1439/2882] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:39.462670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0026df-4888-4fc9-a633-66a6a6410a14 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:39.474216 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0026df-4888-4fc9-a633-66a6a6410a14 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:39.479006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39521195-a07d-4060-a896-f3d753b7651a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:39.479621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521195-a07d-4060-a896-f3d753b7651a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:39.479904 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521195-a07d-4060-a896-f3d753b7651a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:39.480178 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521195-a07d-4060-a896-f3d753b7651a admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:39.480823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39521195-a07d-4060-a896-f3d753b7651a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:39.482869 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39521195-a07d-4060-a896-f3d753b7651a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:39.512146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1444/2883] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:45:39 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:45:39.516926 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95d45-4984-487a-bef5-15670fdbc83a None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:39.528960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e95d45-4984-487a-bef5-15670fdbc83a None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:39.596859 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2e750-6709-4472-b3e4-d35b883e384b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:39.597515 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2e750-6709-4472-b3e4-d35b883e384b admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:39.597802 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2e750-6709-4472-b3e4-d35b883e384b admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:39.598127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2e750-6709-4472-b3e4-d35b883e384b admin admin] PATH_INFO: `/v3/users` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:39.598772 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2e750-6709-4472-b3e4-d35b883e384b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:39.600732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d2e750-6709-4472-b3e4-d35b883e384b admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:39.625428 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1440/2884] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:45:39 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:45:39.629442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39521195-a07d-4060-a896-f3d753b7651a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:39.640621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39521195-a07d-4060-a896-f3d753b7651a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:39.645438 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0a89f-a3a0-401d-b296-6dd98b37e8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:39.646089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0a89f-a3a0-401d-b296-6dd98b37e8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:39.646378 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0a89f-a3a0-401d-b296-6dd98b37e8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:39.646660 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0a89f-a3a0-401d-b296-6dd98b37e8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:39.647335 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0a89f-a3a0-401d-b296-6dd98b37e8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:39.649050 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0a89f-a3a0-401d-b296-6dd98b37e8fe admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:39.661273 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1445/2885] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:45:39.665472 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2e750-6709-4472-b3e4-d35b883e384b admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:39.676262 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d2e750-6709-4472-b3e4-d35b883e384b admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:39.680865 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28407fcf-af27-462c-9f4b-d0192f4170f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:39.681554 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28407fcf-af27-462c-9f4b-d0192f4170f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:39.681837 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28407fcf-af27-462c-9f4b-d0192f4170f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:39.682111 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28407fcf-af27-462c-9f4b-d0192f4170f3 admin admin] PATH_INFO: `/v3/projects/588509197e27499087ec1efe8338d7fc/users/e6ea3efdbeeb43628e78860ccf6d4d5e/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:39.697380 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28407fcf-af27-462c-9f4b-d0192f4170f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=588509197e27499087ec1efe8338d7fc, user_id=e6ea3efdbeeb43628e78860ccf6d4d5e, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:39.700045 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28407fcf-af27-462c-9f4b-d0192f4170f3 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:39.717970 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1441/2886] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:45:39 2022] PUT /identity/v3/projects/588509197e27499087ec1efe8338d7fc/users/e6ea3efdbeeb43628e78860ccf6d4d5e/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:45:39.719839 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0a89f-a3a0-401d-b296-6dd98b37e8fe admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:39.730488 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0a89f-a3a0-401d-b296-6dd98b37e8fe admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:39.772217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:39.793204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:39.793905 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:39.794838 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:39.795193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:39.795975 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:39.797743 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:39.807868 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1446/2887] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:45:39.812246 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28407fcf-af27-462c-9f4b-d0192f4170f3 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:39.822986 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28407fcf-af27-462c-9f4b-d0192f4170f3 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:39.827665 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d8538-443f-46e7-b4f0-2b8795d648f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:39.828325 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d8538-443f-46e7-b4f0-2b8795d648f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:39.828609 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d8538-443f-46e7-b4f0-2b8795d648f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:39.828886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d8538-443f-46e7-b4f0-2b8795d648f7 admin admin] PATH_INFO: `/v3/projects/588509197e27499087ec1efe8338d7fc/users/e6ea3efdbeeb43628e78860ccf6d4d5e/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:39.830974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d8538-443f-46e7-b4f0-2b8795d648f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=588509197e27499087ec1efe8338d7fc, user_id=e6ea3efdbeeb43628e78860ccf6d4d5e, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:39.833710 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d8538-443f-46e7-b4f0-2b8795d648f7 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:39.836984 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:39.847015 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1442/2888] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:45:39 2022] PUT /identity/v3/projects/588509197e27499087ec1efe8338d7fc/users/e6ea3efdbeeb43628e78860ccf6d4d5e/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:45:40.219842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:40.222237 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:40.306288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272dd10-864c-433f-82f6-29941e34d629 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:40.378011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:40.378662 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:40.378932 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:40.379217 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:40.381552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:40.463073 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:40.464713 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:40.476599 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1447/2889] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:44.979886 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a110b4a-10ee-458d-aab7-19cd3f009f57 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:44.981077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a110b4a-10ee-458d-aab7-19cd3f009f57 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:44.981366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a110b4a-10ee-458d-aab7-19cd3f009f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:45.016827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-2a110b4a-10ee-458d-aab7-19cd3f009f57 None None] MFA Rules not processed for user `e6ea3efdbeeb43628e78860ccf6d4d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:45.018642 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2a110b4a-10ee-458d-aab7-19cd3f009f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:45.117544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1443/2890] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:45.131496 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:45.133006 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:45.189221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:45.197339 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6e736-b867-42f5-8cab-81a45f7241bc service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:45.202420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '464190e0ccaf4f82942f518df1bf6c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:45.203134 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:45.203520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:45.204004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:45.274321 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:45.276065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:45.291070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1448/2891] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:45.447960 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33817f-ac4e-4998-b046-6b183dd0afad None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:45.448266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33817f-ac4e-4998-b046-6b183dd0afad None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:45.448548 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33817f-ac4e-4998-b046-6b183dd0afad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:45.469161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-5e33817f-ac4e-4998-b046-6b183dd0afad None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:45.470542 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-5e33817f-ac4e-4998-b046-6b183dd0afad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:45.530659 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1444/2892] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:45:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:45.542863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:45.544338 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:45.612859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399bfd4-c438-4270-9d1d-9f3fa90864b3 service glance] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:45.617832 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:45.618435 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:45.618718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:45.618991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:45.628567 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:45.630258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:45.639928 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1449/2893] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:45:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:45:48.694399 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eeba4-157c-4dad-9a4d-3811719e5765 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:48.694945 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eeba4-157c-4dad-9a4d-3811719e5765 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:48.695113 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eeba4-157c-4dad-9a4d-3811719e5765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:48.719408 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-9d5eeba4-157c-4dad-9a4d-3811719e5765 None None] MFA Rules not processed for user `f548ccf8c5a14b108aa8b851560139d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:48.720779 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9d5eeba4-157c-4dad-9a4d-3811719e5765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:48.734190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1445/2894] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:48.779384 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:48.785076 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:48.802844 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f7deb-411a-4de5-aa07-6e56ea9cefa8 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:48.818876 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8fa402682fd24228a77801edf1bb4d59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:48.819524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:48.819813 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:48.820089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:48.827051 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:48.920542 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:48.922082 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:48.936979 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1450/2895] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:49.333593 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eeba4-157c-4dad-9a4d-3811719e5765 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:49.335339 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-9d5eeba4-157c-4dad-9a4d-3811719e5765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:49.414798 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eeba4-157c-4dad-9a4d-3811719e5765 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:49.418496 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b169c1-2e31-467c-b3d8-f236c6f72acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:49.419449 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b169c1-2e31-467c-b3d8-f236c6f72acb service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:49.419744 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b169c1-2e31-467c-b3d8-f236c6f72acb service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:49.420127 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b169c1-2e31-467c-b3d8-f236c6f72acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:49.421943 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-29b169c1-2e31-467c-b3d8-f236c6f72acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:49.435438 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b169c1-2e31-467c-b3d8-f236c6f72acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:49.436997 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b169c1-2e31-467c-b3d8-f236c6f72acb service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:49.446209 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1446/2896] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:50.035374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:50.061428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555f3e2-4514-4b57-a02c-bda0ca9ee713 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:50.067950 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce571840-d08f-4952-866d-7e4a3153aabd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:50.068996 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce571840-d08f-4952-866d-7e4a3153aabd service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:50.069581 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce571840-d08f-4952-866d-7e4a3153aabd service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:50.070079 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce571840-d08f-4952-866d-7e4a3153aabd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:50.072294 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ce571840-d08f-4952-866d-7e4a3153aabd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:50.098193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce571840-d08f-4952-866d-7e4a3153aabd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:50.099773 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce571840-d08f-4952-866d-7e4a3153aabd service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:50.126521 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1451/2897] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:45:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:45:55.594014 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc91ad-ca3f-4714-9d7e-50b72e861002 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:55.594550 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc91ad-ca3f-4714-9d7e-50b72e861002 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:55.594661 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc91ad-ca3f-4714-9d7e-50b72e861002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:55.614037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-e3dc91ad-ca3f-4714-9d7e-50b72e861002 None None] MFA Rules not processed for user `24f16ea7c1d740ac8de5a4a19b165350`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:55.615242 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-e3dc91ad-ca3f-4714-9d7e-50b72e861002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:55.622281 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1447/2898] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:45:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:57.550960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec90c5b-9653-47ba-a3b9-bca9700df4dd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:57.551536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec90c5b-9653-47ba-a3b9-bca9700df4dd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:57.551536 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec90c5b-9653-47ba-a3b9-bca9700df4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:57.559415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:45:57.574022 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bec90c5b-9653-47ba-a3b9-bca9700df4dd None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:45:57.575969 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bec90c5b-9653-47ba-a3b9-bca9700df4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:45:57.583620 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1452/2899] 10.222.0.58 () {58 vars in 1009 bytes} [Mon Oct 3 13:45:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:45:57.596820 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc91ad-ca3f-4714-9d7e-50b72e861002 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:45:57.609273 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc91ad-ca3f-4714-9d7e-50b72e861002 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:45:57.615220 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e935a-e5a7-4d6f-8082-058ad0fd5918 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:45:57.615850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e935a-e5a7-4d6f-8082-058ad0fd5918 service swift] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:45:57.616138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e935a-e5a7-4d6f-8082-058ad0fd5918 service swift] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:45:57.616447 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e935a-e5a7-4d6f-8082-058ad0fd5918 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:45:57.626062 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e935a-e5a7-4d6f-8082-058ad0fd5918 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:45:57.627664 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e935a-e5a7-4d6f-8082-058ad0fd5918 service swift] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:45:57.636495 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1448/2900] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:45:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:46:07.568113 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:46:07.632827 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=99 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:46:10.524288 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec90c5b-9653-47ba-a3b9-bca9700df4dd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:46:10.534955 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec90c5b-9653-47ba-a3b9-bca9700df4dd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:46:10.538622 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:46:10.539236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:46:10.539520 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:46:10.539798 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:46:10.541718 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:46:10.552670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:46:10.554137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:46:10.562690 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1453/2901] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:46:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:46:15.925917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e935a-e5a7-4d6f-8082-058ad0fd5918 service swift] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:46:15.935769 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e935a-e5a7-4d6f-8082-058ad0fd5918 service swift] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:46:15.948888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f9df2-836f-4c67-a266-d982696aa967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:46:15.949732 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f9df2-836f-4c67-a266-d982696aa967 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:46:15.950975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f9df2-836f-4c67-a266-d982696aa967 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:46:15.951473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f9df2-836f-4c67-a266-d982696aa967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:46:15.953483 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-d02f9df2-836f-4c67-a266-d982696aa967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:46:16.026206 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f9df2-836f-4c67-a266-d982696aa967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:46:16.027663 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f9df2-836f-4c67-a266-d982696aa967 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:46:16.037512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1449/2902] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:46:20.558524 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:46:26.033002 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=30 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:46:29.196182 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:46:29.206329 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a096bc4-70a0-4c91-93f8-6adf94d791c9 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:46:29.210208 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e53112-dcc3-47fa-ae57-db68f1c881bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:46:29.210851 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e53112-dcc3-47fa-ae57-db68f1c881bf service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:46:29.211137 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e53112-dcc3-47fa-ae57-db68f1c881bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:46:29.211416 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e53112-dcc3-47fa-ae57-db68f1c881bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:46:29.221146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e53112-dcc3-47fa-ae57-db68f1c881bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:46:29.222625 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e53112-dcc3-47fa-ae57-db68f1c881bf service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:46:29.232402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1454/2903] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:46:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:46:39.227853 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:09.267850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7f00-cad7-41bf-83c3-bd44c25bcb24 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:09.268492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7f00-cad7-41bf-83c3-bd44c25bcb24 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:09.268492 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a7f00-cad7-41bf-83c3-bd44c25bcb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:09.288054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-b72a7f00-cad7-41bf-83c3-bd44c25bcb24 None None] MFA Rules not processed for user `c1ff08df377e4a83aae3afedd204e1ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:48:09.289275 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b72a7f00-cad7-41bf-83c3-bd44c25bcb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:09.296405 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1450/2904] 10.222.0.58 () {58 vars in 967 bytes} [Mon Oct 3 13:48:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:48:09.337750 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e53112-dcc3-47fa-ae57-db68f1c881bf service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:09.349613 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e53112-dcc3-47fa-ae57-db68f1c881bf service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:09.354827 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aad8a-09a5-422c-97f4-a6ec15489112 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ba4c42b47794c9ba5fc6ee557e2cfc3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:09.355432 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aad8a-09a5-422c-97f4-a6ec15489112 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:09.355712 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aad8a-09a5-422c-97f4-a6ec15489112 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:09.355985 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0aad8a-09a5-422c-97f4-a6ec15489112 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:09.365036 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aad8a-09a5-422c-97f4-a6ec15489112 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:09.366725 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0aad8a-09a5-422c-97f4-a6ec15489112 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:09.375204 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1455/2905] 10.222.0.58 () {60 vars in 1384 bytes} [Mon Oct 3 13:48:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 03 13:48:14.197591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a7f00-cad7-41bf-83c3-bd44c25bcb24 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:14.197591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-b72a7f00-cad7-41bf-83c3-bd44c25bcb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:14.294546 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a7f00-cad7-41bf-83c3-bd44c25bcb24 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:14.300964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:14.301730 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:14.302174 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:14.302512 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:14.304717 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:14.319107 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:14.321359 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:14.333931 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1451/2906] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:48:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:48:17.999011 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aad8a-09a5-422c-97f4-a6ec15489112 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:18.086428 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0aad8a-09a5-422c-97f4-a6ec15489112 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:18.094041 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:18.095381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:18.095688 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:18.095968 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] PATH_INFO: `/v3/users/24f16ea7c1d740ac8de5a4a19b165350` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:18.097381 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f16ea7c1d740ac8de5a4a19b165350)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:18.099695 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:18.158362 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f16ea7c1d740ac8de5a4a19b165350'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.170973 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f16ea7c1d740ac8de5a4a19b165350'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.171729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f16ea7c1d740ac8de5a4a19b165350'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.180301 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f16ea7c1d740ac8de5a4a19b165350'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.189324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f16ea7c1d740ac8de5a4a19b165350'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.196315 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1456/2907] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:48:17 2022] DELETE /identity/v3/users/24f16ea7c1d740ac8de5a4a19b165350 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:18.383739 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:18.403773 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd04a60-126e-4c05-815b-e0bcd9117421 service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:18.468165 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:18.468835 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:18.469123 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:18.469402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] PATH_INFO: `/v3/projects/e8b666774ba8449f90cf329b90725fe4` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:18.471228 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8b666774ba8449f90cf329b90725fe4)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:18.473280 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:18.526058 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8b666774ba8449f90cf329b90725fe4', 'request_id': 'req-2428bbff-99ab-4337-9b32-350cf7edf2a0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.533005 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8b666774ba8449f90cf329b90725fe4', 'request_id': 'req-2428bbff-99ab-4337-9b32-350cf7edf2a0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.540757 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8b666774ba8449f90cf329b90725fe4', 'request_id': 'req-2428bbff-99ab-4337-9b32-350cf7edf2a0'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.557923 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] The token cache is being invalidate because project e8b666774ba8449f90cf329b90725fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:48:18.558218 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.558967 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.560469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1452/2908] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:48:18 2022] DELETE /identity/v3/projects/e8b666774ba8449f90cf329b90725fe4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:18.565681 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:18.567120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:18.581833 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e63dbb-0258-454f-b36e-01cfa2621de4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:18.653309 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:18.653880 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:18.654166 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:18.654443 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] PATH_INFO: `/v3/users/e6ea3efdbeeb43628e78860ccf6d4d5e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:18.655855 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ea3efdbeeb43628e78860ccf6d4d5e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:18.658586 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:18.665214 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:18.740167 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ea3efdbeeb43628e78860ccf6d4d5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.763048 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ea3efdbeeb43628e78860ccf6d4d5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.764269 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ea3efdbeeb43628e78860ccf6d4d5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.776736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ea3efdbeeb43628e78860ccf6d4d5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.788255 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-12c4878f-8c8a-47e7-b3a5-66b6663aef06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ea3efdbeeb43628e78860ccf6d4d5e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:18.796811 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1457/2909] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:48:18 2022] DELETE /identity/v3/users/e6ea3efdbeeb43628e78860ccf6d4d5e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:19.021064 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:19.023688 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:19.050803 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428bbff-99ab-4337-9b32-350cf7edf2a0 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:19.065061 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:19.124477 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:19.125247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.125584 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.125863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] PATH_INFO: `/v3/projects/588509197e27499087ec1efe8338d7fc` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.127975 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=588509197e27499087ec1efe8338d7fc)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:19.130012 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:19.187469 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588509197e27499087ec1efe8338d7fc', 'request_id': 'req-410b1edf-5445-4328-9ad6-99e54c655942'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:19.200180 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588509197e27499087ec1efe8338d7fc', 'request_id': 'req-410b1edf-5445-4328-9ad6-99e54c655942'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:19.206790 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588509197e27499087ec1efe8338d7fc', 'request_id': 'req-410b1edf-5445-4328-9ad6-99e54c655942'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:19.221296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] The token cache is being invalidate because project 588509197e27499087ec1efe8338d7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:48:19.221623 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:19.222467 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-410b1edf-5445-4328-9ad6-99e54c655942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:19.224067 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1453/2910] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:48:19 2022] DELETE /identity/v3/projects/588509197e27499087ec1efe8338d7fc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:19.229314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470528b-f773-4125-9172-7c4199a9fd77 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.229835 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470528b-f773-4125-9172-7c4199a9fd77 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.229938 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470528b-f773-4125-9172-7c4199a9fd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.253085 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d470528b-f773-4125-9172-7c4199a9fd77 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:48:19.254509 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d470528b-f773-4125-9172-7c4199a9fd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:19.327569 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1458/2911] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:48:19.336073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0012c-d1c9-4330-92c9-27335c24fc50 None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.336369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0012c-d1c9-4330-92c9-27335c24fc50 None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.336650 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0012c-d1c9-4330-92c9-27335c24fc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.358473 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-56a0012c-d1c9-4330-92c9-27335c24fc50 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:48:19.359747 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-56a0012c-d1c9-4330-92c9-27335c24fc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:19.369513 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1454/2912] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:48:19.373997 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470528b-f773-4125-9172-7c4199a9fd77 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:19.384423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470528b-f773-4125-9172-7c4199a9fd77 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:19.389177 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:19.389831 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.390120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.390402 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.391070 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 03 13:48:19.391383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:19.393737 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:19.410564 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1459/2913] 10.222.0.58 () {58 vars in 1153 bytes} [Mon Oct 3 13:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 03 13:48:19.412037 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0012c-d1c9-4330-92c9-27335c24fc50 None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:19.421701 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0012c-d1c9-4330-92c9-27335c24fc50 None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:19.427183 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf69fb-97b8-42e5-8c7c-a16494e88c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:19.427832 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf69fb-97b8-42e5-8c7c-a16494e88c73 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.428122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf69fb-97b8-42e5-8c7c-a16494e88c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.428537 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf69fb-97b8-42e5-8c7c-a16494e88c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.429199 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf69fb-97b8-42e5-8c7c-a16494e88c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:19.431149 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf69fb-97b8-42e5-8c7c-a16494e88c73 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:19.459518 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1455/2914] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:48:19 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:48:19.463784 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:19.475210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb628402-feb0-4317-bce3-c2aeba8a69e1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:19.540130 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225fafd-37c7-409f-b27a-17c61ed37213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:19.540766 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225fafd-37c7-409f-b27a-17c61ed37213 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.541052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225fafd-37c7-409f-b27a-17c61ed37213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.541383 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225fafd-37c7-409f-b27a-17c61ed37213 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.542064 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225fafd-37c7-409f-b27a-17c61ed37213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:19.544419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e225fafd-37c7-409f-b27a-17c61ed37213 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:19.569444 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1460/2915] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:48:19 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:48:19.573735 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf69fb-97b8-42e5-8c7c-a16494e88c73 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:19.585360 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf69fb-97b8-42e5-8c7c-a16494e88c73 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:19.590148 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d0383-7f3d-44c1-8427-0d77f276d712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:19.590756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d0383-7f3d-44c1-8427-0d77f276d712 admin admin] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.591054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d0383-7f3d-44c1-8427-0d77f276d712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.591315 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d0383-7f3d-44c1-8427-0d77f276d712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.592161 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d0383-7f3d-44c1-8427-0d77f276d712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:19.593831 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d0383-7f3d-44c1-8427-0d77f276d712 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:19.605750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1456/2916] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:48:19.609991 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225fafd-37c7-409f-b27a-17c61ed37213 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:19.616427 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:19.623133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225fafd-37c7-409f-b27a-17c61ed37213 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:19.629942 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf3d90-0c18-4ea8-b123-86e464d1a13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:19.630561 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf3d90-0c18-4ea8-b123-86e464d1a13c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.630859 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf3d90-0c18-4ea8-b123-86e464d1a13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.631125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf3d90-0c18-4ea8-b123-86e464d1a13c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.631779 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf3d90-0c18-4ea8-b123-86e464d1a13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:19.633415 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf3d90-0c18-4ea8-b123-86e464d1a13c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:19.646781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1461/2917] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:48:19.650949 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d0383-7f3d-44c1-8427-0d77f276d712 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:19.662974 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d0383-7f3d-44c1-8427-0d77f276d712 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:19.667678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:19.668348 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.668635 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.668913 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] PATH_INFO: `/v3/projects/0ccd8015806a42bc81a8cbe3d2cab8d7/users/3de6ff608b8544ef9a1130666b7c3f14/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.685290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccd8015806a42bc81a8cbe3d2cab8d7, user_id=3de6ff608b8544ef9a1130666b7c3f14, role_id=b19ca4b95a9f4d6eaad2f86da66e5aa8)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:19.687905 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:19.701668 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1457/2918] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:48:19 2022] PUT /identity/v3/projects/0ccd8015806a42bc81a8cbe3d2cab8d7/users/3de6ff608b8544ef9a1130666b7c3f14/roles/b19ca4b95a9f4d6eaad2f86da66e5aa8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:19.708792 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee0eed-8ae3-470d-86b4-4bed02e77a05 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.709089 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee0eed-8ae3-470d-86b4-4bed02e77a05 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.709373 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee0eed-8ae3-470d-86b4-4bed02e77a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.729299 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-46ee0eed-8ae3-470d-86b4-4bed02e77a05 None None] MFA Rules not processed for user `72593536090c47589cfad801ff3da440`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:48:19.730609 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-46ee0eed-8ae3-470d-86b4-4bed02e77a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:19.791584 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1462/2919] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 03 13:48:19.801324 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:19.803138 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:19.847189 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:19.862366 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea95087-8983-4ff0-a0b6-ea4d6a319829 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:19.866032 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563c841-7426-4f49-a968-2a8a572b6c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:19.866653 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c841-7426-4f49-a968-2a8a572b6c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:19.866962 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c841-7426-4f49-a968-2a8a572b6c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:19.867215 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563c841-7426-4f49-a968-2a8a572b6c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:19.876778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563c841-7426-4f49-a968-2a8a572b6c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:19.878190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563c841-7426-4f49-a968-2a8a572b6c3d service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:19.887652 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1458/2920] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:48:24.674791 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dc8df-c284-44c4-a6b2-28d8598e7dff None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:24.675353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dc8df-c284-44c4-a6b2-28d8598e7dff None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:24.675353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dc8df-c284-44c4-a6b2-28d8598e7dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:24.712104 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0b2dc8df-c284-44c4-a6b2-28d8598e7dff None None] MFA Rules not processed for user `3de6ff608b8544ef9a1130666b7c3f14`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:48:24.713389 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0b2dc8df-c284-44c4-a6b2-28d8598e7dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:24.805920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1463/2921] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:48:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:48:24.806946 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563c841-7426-4f49-a968-2a8a572b6c3d service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:24.817891 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563c841-7426-4f49-a968-2a8a572b6c3d service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:24.826394 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d12b-abb5-4a44-a93d-b0fcb03b0643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:24.827205 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d12b-abb5-4a44-a93d-b0fcb03b0643 admin admin] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:24.827425 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d12b-abb5-4a44-a93d-b0fcb03b0643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:24.827777 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d12b-abb5-4a44-a93d-b0fcb03b0643 admin admin] PATH_INFO: `/v3/projects` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:24.828649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9d12b-abb5-4a44-a93d-b0fcb03b0643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:24.831327 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9d12b-abb5-4a44-a93d-b0fcb03b0643 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:24.868538 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1459/2922] 10.222.0.58 () {60 vars in 1153 bytes} [Mon Oct 3 13:48:24 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 03 13:48:24.871314 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dc8df-c284-44c4-a6b2-28d8598e7dff None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:24.889862 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d12b-abb5-4a44-a93d-b0fcb03b0643 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:24.892598 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dc8df-c284-44c4-a6b2-28d8598e7dff None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:24.961984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4065fb-7a53-41df-b5f2-ece9daa9964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:24.962217 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d12b-abb5-4a44-a93d-b0fcb03b0643 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:24.962817 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4065fb-7a53-41df-b5f2-ece9daa9964d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:24.963240 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4065fb-7a53-41df-b5f2-ece9daa9964d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:24.963382 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4065fb-7a53-41df-b5f2-ece9daa9964d admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:24.964181 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4065fb-7a53-41df-b5f2-ece9daa9964d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:24.966161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4065fb-7a53-41df-b5f2-ece9daa9964d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:24.966648 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:24.967350 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:24.967707 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:24.967989 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:24.969839 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:24.983365 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:24.985678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:24.994238 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1464/2923] 10.222.0.58 () {60 vars in 1144 bytes} [Mon Oct 3 13:48:24 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 03 13:48:24.995296 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4065fb-7a53-41df-b5f2-ece9daa9964d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:24.996530 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1460/2924] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:48:25.006268 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4065fb-7a53-41df-b5f2-ece9daa9964d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:25.011015 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57647-8632-4da5-a5a6-caecd42e6b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:25.011627 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57647-8632-4da5-a5a6-caecd42e6b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:25.011915 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57647-8632-4da5-a5a6-caecd42e6b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:25.012194 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57647-8632-4da5-a5a6-caecd42e6b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:25.013445 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c57647-8632-4da5-a5a6-caecd42e6b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:25.015105 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c57647-8632-4da5-a5a6-caecd42e6b55 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:25.021114 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:25.026653 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1465/2925] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:48:25.035414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:25.047109 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b92a-bd93-4fc3-9e98-307514cefd65 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:25.052750 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2adf6-fdcd-4726-8c60-bfe3bd12861c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:25.053433 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2adf6-fdcd-4726-8c60-bfe3bd12861c admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:25.054009 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2adf6-fdcd-4726-8c60-bfe3bd12861c admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:25.054290 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2adf6-fdcd-4726-8c60-bfe3bd12861c admin admin] PATH_INFO: `/v3/projects/b837637fc3484a17b3ef86c29bca3e83/users/de714dbd4f234369870b39f8cec6b017/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:25.071263 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2adf6-fdcd-4726-8c60-bfe3bd12861c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b837637fc3484a17b3ef86c29bca3e83, user_id=de714dbd4f234369870b39f8cec6b017, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:25.073914 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2adf6-fdcd-4726-8c60-bfe3bd12861c admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:25.088850 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1461/2926] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:48:25 2022] PUT /identity/v3/projects/b837637fc3484a17b3ef86c29bca3e83/users/de714dbd4f234369870b39f8cec6b017/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:25.092913 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57647-8632-4da5-a5a6-caecd42e6b55 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:25.104168 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57647-8632-4da5-a5a6-caecd42e6b55 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:25.170730 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:25.171374 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:25.171739 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:25.172025 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:25.173258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:25.175420 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:25.186231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1466/2927] 10.222.0.58 () {58 vars in 1122 bytes} [Mon Oct 3 13:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 03 13:48:25.191414 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2adf6-fdcd-4726-8c60-bfe3bd12861c admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:25.201708 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2adf6-fdcd-4726-8c60-bfe3bd12861c admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:25.207054 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fc9e0-4143-4497-826e-59de229ee32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:25.207725 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fc9e0-4143-4497-826e-59de229ee32a admin admin] REQUEST_METHOD: `PUT` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:25.208004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fc9e0-4143-4497-826e-59de229ee32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:25.208279 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fc9e0-4143-4497-826e-59de229ee32a admin admin] PATH_INFO: `/v3/projects/b837637fc3484a17b3ef86c29bca3e83/users/de714dbd4f234369870b39f8cec6b017/roles/cdb47dc2f693403f958bbca4dab63130` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:25.210222 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fc9e0-4143-4497-826e-59de229ee32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b837637fc3484a17b3ef86c29bca3e83, user_id=de714dbd4f234369870b39f8cec6b017, role_id=cdb47dc2f693403f958bbca4dab63130)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:25.212900 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fc9e0-4143-4497-826e-59de229ee32a admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:25.224411 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1462/2928] 10.222.0.58 () {60 vars in 1483 bytes} [Mon Oct 3 13:48:25 2022] PUT /identity/v3/projects/b837637fc3484a17b3ef86c29bca3e83/users/de714dbd4f234369870b39f8cec6b017/roles/cdb47dc2f693403f958bbca4dab63130 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:28.998232 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:29.001667 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:29.221952 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee9a76-96c1-4059-a09f-9a999b16dae4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:29.349729 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:29.350704 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:29.351767 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:29.352161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:29.354556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:29.372805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:29.375037 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:29.388680 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1467/2929] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:48:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:48:31.545737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752008c8-8ad6-4951-a92a-cac76b40508d None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:31.546224 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752008c8-8ad6-4951-a92a-cac76b40508d None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:31.546341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752008c8-8ad6-4951-a92a-cac76b40508d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:31.580709 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-752008c8-8ad6-4951-a92a-cac76b40508d None None] MFA Rules not processed for user `de714dbd4f234369870b39f8cec6b017`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:48:31.582004 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-752008c8-8ad6-4951-a92a-cac76b40508d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:31.665857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1463/2930] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:48:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:48:31.681260 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:31.682897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:31.722489 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:31.770507 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10312d-6b79-4b5b-9d73-68c0a7817613 service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:31.776535 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:31.777274 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:31.777781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:31.778133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:31.870275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:31.871982 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:31.894834 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1468/2931] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:48:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:48:33.288157 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a05fd5-e0b6-4772-bc1e-4d687a791f1e None None] REQUEST_METHOD: `POST` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:33.288711 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a05fd5-e0b6-4772-bc1e-4d687a791f1e None None] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:33.288863 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a05fd5-e0b6-4772-bc1e-4d687a791f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:33.309003 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.auth.core [None req-c2a05fd5-e0b6-4772-bc1e-4d687a791f1e None None] MFA Rules not processed for user `3de6ff608b8544ef9a1130666b7c3f14`. Rule list: `[]` (Enabled: `True`). {{(pid=81423) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 03 13:48:33.310266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-c2a05fd5-e0b6-4772-bc1e-4d687a791f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:33.317738 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1464/2932] 10.222.0.58 () {58 vars in 958 bytes} [Mon Oct 3 13:48:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 03 13:48:34.432687 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:34.435157 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:34.520161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4a2c2-001d-4cda-b08f-d3751ec417d4 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:34.524897 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:34.525602 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:34.525854 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:34.526133 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] PATH_INFO: `/v3/users/f548ccf8c5a14b108aa8b851560139d0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:34.527280 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f548ccf8c5a14b108aa8b851560139d0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:34.529228 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:34.593120 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f548ccf8c5a14b108aa8b851560139d0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:34.607264 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f548ccf8c5a14b108aa8b851560139d0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:34.608018 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f548ccf8c5a14b108aa8b851560139d0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:34.611161 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:34.619023 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f548ccf8c5a14b108aa8b851560139d0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:34.632155 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f548ccf8c5a14b108aa8b851560139d0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:34.642606 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1469/2933] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:48:34 2022] DELETE /identity/v3/users/f548ccf8c5a14b108aa8b851560139d0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:34.837985 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a05fd5-e0b6-4772-bc1e-4d687a791f1e None None] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:34.846678 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:34.851971 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a05fd5-e0b6-4772-bc1e-4d687a791f1e None None] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:34.921461 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:34.922122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:34.922675 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:34.922888 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] PATH_INFO: `/v3/projects/0e005cbca8874c76bf8ad8f5d336b189` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:34.924721 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e005cbca8874c76bf8ad8f5d336b189)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:34.927132 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:34.994055 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e005cbca8874c76bf8ad8f5d336b189', 'request_id': 'req-10cb2557-fc86-4d62-951f-9d4f9989e637'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.009978 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e005cbca8874c76bf8ad8f5d336b189', 'request_id': 'req-10cb2557-fc86-4d62-951f-9d4f9989e637'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.016520 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e005cbca8874c76bf8ad8f5d336b189', 'request_id': 'req-10cb2557-fc86-4d62-951f-9d4f9989e637'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.031298 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] The token cache is being invalidate because project 0e005cbca8874c76bf8ad8f5d336b189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:48:35.032797 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.033499 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.035141 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1465/2934] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:48:34 2022] DELETE /identity/v3/projects/0e005cbca8874c76bf8ad8f5d336b189 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:35.044221 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:35.045946 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:35.062332 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce24057-9d80-4146-99cb-0b63b9ef6df4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:35.129088 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:35.129762 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:35.130046 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:35.130324 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] PATH_INFO: `/v3/users/66354734d9ae4e6dadbd2d59c1c4c9b1` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:35.131529 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] RBAC: Authorizing `identity:delete_user(user_id=66354734d9ae4e6dadbd2d59c1c4c9b1)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:35.133639 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:35.187371 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66354734d9ae4e6dadbd2d59c1c4c9b1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.200190 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66354734d9ae4e6dadbd2d59c1c4c9b1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.200917 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66354734d9ae4e6dadbd2d59c1c4c9b1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.209425 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66354734d9ae4e6dadbd2d59c1c4c9b1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.218736 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66354734d9ae4e6dadbd2d59c1c4c9b1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.227187 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1470/2935] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:48:35 2022] DELETE /identity/v3/users/66354734d9ae4e6dadbd2d59c1c4c9b1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:35.423848 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:35.425395 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:35.439829 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb2557-fc86-4d62-951f-9d4f9989e637 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:35.504266 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:35.504922 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:35.505190 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:35.505489 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] PATH_INFO: `/v3/projects/ce434d4dbdbd420cbeb39478f2dbb3f0` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:35.506607 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce434d4dbdbd420cbeb39478f2dbb3f0)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:35.508544 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:35.552461 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:35.570573 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce434d4dbdbd420cbeb39478f2dbb3f0', 'request_id': 'req-2fe2dd4e-6773-481f-97a2-77241940fe15'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.577388 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce434d4dbdbd420cbeb39478f2dbb3f0', 'request_id': 'req-2fe2dd4e-6773-481f-97a2-77241940fe15'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.583756 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce434d4dbdbd420cbeb39478f2dbb3f0', 'request_id': 'req-2fe2dd4e-6773-481f-97a2-77241940fe15'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.597911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] The token cache is being invalidate because project ce434d4dbdbd420cbeb39478f2dbb3f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81423) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:48:35.598244 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.598855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:48:35.600713 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1466/2936] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:48:35 2022] DELETE /identity/v3/projects/ce434d4dbdbd420cbeb39478f2dbb3f0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:48:45.222610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=11,keystone:SELECT=59,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:45.594122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:50.514405 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:50.516007 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:50.641823 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790ced6-503f-4da3-8a3c-4ca8710a223f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:50.702146 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:50.702761 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:50.703047 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:50.703327 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:50.705080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:50.716995 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:50.718621 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:50.729537 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1471/2937] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:48:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:48:51.205792 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:51.207321 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:51.244427 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2dd4e-6773-481f-97a2-77241940fe15 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:51.249894 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:51.250503 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:51.250785 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:51.251060 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:51.252828 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:51.265567 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:51.267156 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:51.278167 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1467/2938] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:48:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:48:53.790419 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:48:53.791960 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:53.847306 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:48:53.879951 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33ad5c-6e95-4cd9-ad69-1d82793f2121 service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:48:53.889505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:48:53.889505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:48:53.889505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:48:53.889505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:48:53.891483 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:48:53.911118 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:48:53.912998 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:48:53.925172 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1472/2939] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:48:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:49:01.273855 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=15 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:49:03.919664 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=14 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:49:04.805937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:49:04.807649 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:49:04.845841 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0d6c5-099a-4ce8-b65a-317dc6648b68 service placement] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:49:04.852073 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3ece3405a2b43288136ddb79b84a106', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:49:04.852684 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:49:04.852963 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:49:04.853236 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:49:04.854994 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:49:04.868917 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:49:04.870604 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:49:04.882616 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1468/2940] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:49:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:49:07.259099 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:49:07.260546 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:49:07.346644 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da956-008f-47ec-bc7d-600f9135e7bb service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:49:07.351193 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:49:07.351842 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:49:07.352129 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:49:07.352411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:49:07.354276 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:49:07.418781 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:49:07.420210 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:49:07.429863 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1473/2941] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:49:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:49:14.877416 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=15 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:49:17.425377 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=63 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:51:08.221131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:51:08.232122 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80e794-1346-4c17-a129-f60de4a026d4 service nova] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:51:08.235795 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:51:08.236402 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:51:08.236686 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:51:08.236964 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:51:08.238788 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:51:08.274737 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:51:08.276211 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:51:08.297631 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1469/2942] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:51:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:51:09.482019 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:51:09.483505 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:51:09.508534 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328556e7-3793-481b-96e1-70001cfacc60 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:51:09.512222 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:51:09.512828 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:51:09.513125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:51:09.513398 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:51:09.515171 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:51:09.529711 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:51:09.531149 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:51:09.542616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1474/2943] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:51:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:51:18.294874 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=18 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:51:18.374722 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:51:18.383591 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f39860-4689-4548-b03e-ca06ef6fe8b2 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:51:18.387436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:51:18.388053 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:51:18.388341 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:51:18.388618 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:51:18.390390 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:51:18.400728 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:51:18.402439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:51:18.411353 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1470/2944] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:51:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:51:19.537259 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=11 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:51:21.513126 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:51:21.522143 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcebb2-13e9-48ed-9fbe-b5371c05dff8 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:51:21.526528 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:51:21.527231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:51:21.527778 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:51:21.527984 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:51:21.529927 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:51:21.541119 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:51:21.542522 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:51:21.551552 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1475/2945] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:51:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:51:21.672143 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:51:21.683358 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e573a-a2e4-4936-9888-98f3a7dcb1a1 service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:51:21.688937 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:51:21.689833 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:51:21.690247 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:51:21.690627 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:51:21.692909 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:51:21.704367 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:51:21.705857 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:51:21.715436 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1471/2946] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:51:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:51:21.810236 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:51:21.819649 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40049029-c194-42ea-8efc-12dc19c89c07 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:51:21.824760 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf865cb3c7fc432884e4e308c397c83d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:51:21.825411 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:51:21.825882 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:51:21.826174 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:51:21.828258 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:51:21.844558 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:51:21.846075 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:51:21.858250 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1476/2947] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:51:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 03 13:51:31.710911 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=12 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:51:31.852670 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=12 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:53:07.637915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:53:07.637915 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:53:07.736466 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f47af-c911-4f79-9336-1de07e59259e service neutron] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:53:07.741583 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:53:07.742201 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:53:07.742528 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:53:07.742933 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] PATH_INFO: `/v3/users/3de6ff608b8544ef9a1130666b7c3f14` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:53:07.744352 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de6ff608b8544ef9a1130666b7c3f14)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:53:07.746335 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:53:07.793208 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de6ff608b8544ef9a1130666b7c3f14'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:07.793892 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de6ff608b8544ef9a1130666b7c3f14'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:07.794505 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de6ff608b8544ef9a1130666b7c3f14'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:07.801313 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de6ff608b8544ef9a1130666b7c3f14'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:07.813264 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de6ff608b8544ef9a1130666b7c3f14'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:07.827011 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1472/2948] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:53:07 2022] DELETE /identity/v3/users/3de6ff608b8544ef9a1130666b7c3f14 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:53:07.995679 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:53:08.006756 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b3dd6-8825-43a6-a7bf-52519cc5baba service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:53:08.065805 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:53:08.066442 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:53:08.066728 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:53:08.067004 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] PATH_INFO: `/v3/projects/0ccd8015806a42bc81a8cbe3d2cab8d7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:53:08.068125 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ccd8015806a42bc81a8cbe3d2cab8d7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:53:08.070112 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:53:08.117043 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ccd8015806a42bc81a8cbe3d2cab8d7', 'request_id': 'req-de04e19b-cdef-440b-a1a6-343953d63136'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.123131 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ccd8015806a42bc81a8cbe3d2cab8d7', 'request_id': 'req-de04e19b-cdef-440b-a1a6-343953d63136'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.129920 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ccd8015806a42bc81a8cbe3d2cab8d7', 'request_id': 'req-de04e19b-cdef-440b-a1a6-343953d63136'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.141215 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] The token cache is being invalidate because project 0ccd8015806a42bc81a8cbe3d2cab8d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:53:08.141556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.142224 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.144080 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1477/2949] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:53:07 2022] DELETE /identity/v3/projects/0ccd8015806a42bc81a8cbe3d2cab8d7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:53:08.148229 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:53:08.150357 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:53:08.163439 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319d0a5-51b6-4bd5-ae28-2eb60c102f7f admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:53:08.214102 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:53:08.224470 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:53:08.225131 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:53:08.225830 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:53:08.226369 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] PATH_INFO: `/v3/users/de714dbd4f234369870b39f8cec6b017` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:53:08.228136 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de714dbd4f234369870b39f8cec6b017)` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:53:08.230093 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:53:08.277864 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de714dbd4f234369870b39f8cec6b017'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.278543 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de714dbd4f234369870b39f8cec6b017'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.279187 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de714dbd4f234369870b39f8cec6b017'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.286817 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de714dbd4f234369870b39f8cec6b017'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.299077 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.notifications [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de714dbd4f234369870b39f8cec6b017'} {{(pid=81423) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.313088 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1473/2950] 10.222.0.58 () {58 vars in 1224 bytes} [Mon Oct 3 13:53:08 2022] DELETE /identity/v3/users/de714dbd4f234369870b39f8cec6b017 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:53:08.487554 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:53:08.489065 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:53:08.502610 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04e19b-cdef-440b-a1a6-343953d63136 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:53:08.562140 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72593536090c47589cfad801ff3da440', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cf1b6fdf131f4a598a5705727a85c56c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:53:08.562775 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:53:08.563061 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:53:08.563343 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] PATH_INFO: `/v3/projects/b837637fc3484a17b3ef86c29bca3e83` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:53:08.564465 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b837637fc3484a17b3ef86c29bca3e83)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:53:08.566423 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:53:08.602231 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:53:08.615556 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b837637fc3484a17b3ef86c29bca3e83', 'request_id': 'req-b40ce851-ebf1-44bc-85e6-511ba24f4009'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.621353 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b837637fc3484a17b3ef86c29bca3e83', 'request_id': 'req-b40ce851-ebf1-44bc-85e6-511ba24f4009'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.628616 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b837637fc3484a17b3ef86c29bca3e83', 'request_id': 'req-b40ce851-ebf1-44bc-85e6-511ba24f4009'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.640290 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] The token cache is being invalidate because project b837637fc3484a17b3ef86c29bca3e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 03 13:53:08.640587 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.641275 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b40ce851-ebf1-44bc-85e6-511ba24f4009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 03 13:53:08.643052 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1478/2951] 10.222.0.58 () {58 vars in 1233 bytes} [Mon Oct 3 13:53:08 2022] DELETE /identity/v3/projects/b837637fc3484a17b3ef86c29bca3e83 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 03 13:53:18.309504 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:53:18.637254 dv-cinder-860090-1-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 03 13:53:56.700793 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] Authenticating user token {{(pid=81423) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 03 13:53:56.702227 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:53:56.776690 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64183c8c-5857-49f2-aaa7-729f6badcc69 admin admin] Validating token access rules against request {{(pid=81423) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 03 13:53:56.835683 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4646f-7f6f-40ee-8d8a-5189560f2c64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bca29b8758b408da2bf05face26b551', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '272f7bf825ad43a7a82311d1db6ae41e', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81423) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 03 13:53:56.836296 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4646f-7f6f-40ee-8d8a-5189560f2c64 service placement] REQUEST_METHOD: `GET` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 03 13:53:56.836576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4646f-7f6f-40ee-8d8a-5189560f2c64 service placement] SCRIPT_NAME: `/identity` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 03 13:53:56.836851 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4646f-7f6f-40ee-8d8a-5189560f2c64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81423) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 03 13:53:56.838934 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.fernet_utils [None req-03a4646f-7f6f-40ee-8d8a-5189560f2c64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81423) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 03 13:53:56.849973 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4646f-7f6f-40ee-8d8a-5189560f2c64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 03 13:53:56.851541 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4646f-7f6f-40ee-8d8a-5189560f2c64 service placement] RBAC: Authorization granted {{(pid=81423) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 03 13:53:56.861778 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: [pid: 81423|app: 0|req: 1474/2952] 10.222.0.58 () {60 vars in 1365 bytes} [Mon Oct 3 13:53:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 03 13:54:06.857576 dv-cinder-860090-1-iscsi devstack@keystone.service[81423]: DEBUG dbcounter [-] [81423] Writing DB stats keystone:SELECT=58 {{(pid=81423) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}